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ABSTRACT 


This report describes the design and use of the pilot study 
concept as a preliminary step in directing Navy program evaluations. 
This methodology focused specifically upon the Navy Leadership and 
Management Education and Training Program [LMET] to identify relevant 
effectiveness issues to provide program manaqers necessary guidance for 
overall program evaluation. Interviews of a cross-section of 51 LMET 
graduates, their immediate supervisor and subordinate were conducted in 
an effort to determine leadership/management improvement. Results 
indicate specific recommendations concerning; the use of pilot studies, 
the interviewing process and LMET evaluation. 
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I. INTRODUCTION 


In late 1978, the U. S. Navy adopted a massive training program for 

its personnel in leadership and management. The program is planned to 

systematically replaced all existing leadership-management courses of 

instruction. Its implementation was Navy-wide with first priority for 

school quotas given to personnel assigned to operational fleet units. 

As the only leadership-management program in the Navy, it was large and 

applied to all Naval levels of command. Due to its widespread 

application, the program required large resource expenditures in time, 

manpower and money. Manpower requirements were expended in students 

attending the ten day course as well as instructor and administrative 

time. The overall goals of the program known as Leadership and 

Management Educating and Training [LMET] were: 

To provide a formal and systematic training program for 
professional development of Navy leaders at critical points 
in their careers, based on research of effective Navy leader¬ 
ship. 

To train officers and petty officers in the specific leader¬ 
ship and management skills needed to perform effectively at 
their level in the chain of command. 

To conduct ongoing evaluation for improving and updating these 
programs. 

To encourage Navy leaders to take personal responsibility for 
implementing effective leadership skills, by means of an edu¬ 
cational approach that emphasizes individual initiative and 
accountability for effective performance as a Navy leader. 
[HRMC.N.O.,p.7] 

The cost of the program was to be offset through "increasing the Navy's 
ability to achieve its overall mission by increasing the effectiveness 
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of Navy leadership across all levels of the chain of command." 
[CNET,N.D.,p.5] 

The Navy has, and continues to invest time, manpower resources and 
money into this program. Yet, there has been to date little effort de¬ 
voted to evaluation of its effectiveness or results. The question of 
where does the program stand today, remains unanswered. Recent 
comments of LMET instructors concerning the course vary, typically: 

The course if not fun to teach.the [teacher] burnout 

ratio is high. 

It will raise the average guy or top performer_it won't 

work for the below average performer, we've lost seme. 

It is a seed planting evolution with attempts at behaviorial 
change. 

You have to sacrifice one [student] for the good learning of 
twenty-eight others. 

You can't tell a student he is wrong....[You can] hope to 
make him see what his affect is on others. 

Additionally, the question arises, what is the perceived value of the 

LMET Program to managers and leaders? Recently interviewed graduates 

of the LMET course had these remarks concerning their training: 

Before LMET I didn't know which way to go.LMET produces 

immediate results.it really helped me. 

LMET probably helped this guy more then any other professional 
school in his career. 

If the decision was mine I wouldn't send anybody to it [LMET 
School]. 

It [LMET School] showed me things that were not clear be¬ 
fore.it helped me to understand myself. 

If the command doesn't support the training [LMET School] the 

man receives then I wouldn't send anyone else.that is 

more detremental than not sending the guy at all. 
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Clearly, there is disagreement among those teaching and implementing, as 
well as those receiving the training. 

The Navy in the late 1960's and early 1970's was confronted by a 
number of contemporary problems that were indicative of the turmoil 
within society as a whole. These problems included drug and alcohol 
abuse, high rates of absenteeism, low retention rates of skilled tech¬ 
nicians, high attrition of newly recruited personnel, and an atmosphere 
of crisis management. It was difficult to link any given problem to a 
simple cause and effect relationship. Rather, the problems appeared to 
be intertwined around the issue of ineffective leadership and manage¬ 
ment at all levels within the chain-of-command. It was imperative that 
action be taken to reverse this situation and it was to that end that the 
Navy turned to the LMET Program. 

It is now mid-1981 with the LMET Program ending its third year. As 
with any large program, LMET was designed to achieve certain goals while 
solving specific problems. In order to determine if LMET has done this, 
that is to judge its effectiveness, it is necessary to evaluate it. Such 
an evaluation would determine if the program is accomplishing its stated 
goals. In fact, one of the goals of the program specifically called for 
ongoing evaluation and updating. [CNET,N.D.] The developer of the 
program, McBer and Company, also recommended an on going evaluation. With 
an evaluation. Navy leaders could assess its health thereby assuring that 
it remain an ongoing program, meeting its goals within the Navy. This 
evaluation is especially important to program managers to enable them to 
judge its cost effectiveness during these times of austere funding. De¬ 
termining whether the LMET Program is "increasing the effectiveness of 
Navy leaders is, however, no simple task. 
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Evaluation of any large. Navy-wide program, such as LMET, is in 
itself an expensive undertaking in terms of budgetary considerations as 
well as time and manpower requirements. For this reason the results must 
be valid and relevant to the program being evaluated. Before embarking 
upon such a task as program evaluation, decision makers need some 
assurance that the evaluation will in fact provide relevant data about 
the program under examination. In addition, the necessity to conduct an 
efficient yet thorough evaluation further compounds the decision maker's 
dilemma. One method of beginning such an evaluation is through the use 
of a pilot study. The concept is basically for an organization, such as 
the Navy, to use some of its own assets to conduct a mini-evaluation of 
the program. This Pilot Study could test the hypothesis, questions, and 
methodology with which a larger, full scaled evaluation may be conducted. 
In evaluating the pilot study outcome, revisions or changes can be made 
prior to commencing the much costlier evaluation. 

This then is the focus of this research, to conduct a pilot study 
of the Navy's LMET Program, thereby assisting in the development of a 
larger, full scale evaluation. The pilot study was undertaken by two 
Naval officer graduate students using a design technique and strategy 
developed with the assistance of McBer and Company and past experience. 
Data from the study was used to provide a rough, first-cut evaluation of 
the LMET Program and more importantly, an evaluation of the methodology 
used. This information can then be applied by the sponsor of this study 
to design a full scale evaluation of the LMET Program. 











The thesis is divided into six major chapters; the first chapter is 
the introduction. The second is a brief history of the evolution of 
leadership philosophy and development within the Navy, with specific 
background on the development of the Navy LMET Program. Emphasis is 
placed on Dr. D. McClelland and McBer and Company's concepts on motive 
acquisition and competency identification. The LMET Program is detailed 
with additional thoughts offered on the need for its evaluation. The 
third chapter discusses those issues and problems anticipated in the 
design of the pilot study itself including the methodology that was used. 
Next is a description of the actual pilot study that was conducted, 
including a discussion on preparations for data gathering, 
pre-arrangements conducted with respondents and the method used to 
analyze data. In simple terms it is what happened and when, during the 
pilot study. The fifth chapter is the analysis and findings of the Pilot 
Study research. It is divided into two major areas, that of the 
interview methodology [process] and a rough analysis of the data 
[content] as it applies to LMET specifically. The final chapter is an 
assessment of the Pilot Study concept, with recommendations directed 
toward a full scale LMET evaluation. 







II. HISTORICAL BACKGROUND OF NAVY LEADERSHIP AND TRAINING 


A. HISTORICAL REVIEW OF NAVY LEADERSHIP 

Since the birth of the United States Navy, senior officers have 
recognized the ability of certain skilled leaders to motivate subordin¬ 
ates in achieving seemingly unattainable goals. These were the leaders 
who were called upon in time of need for accomplishment of those tasks to 
which others had failed. They were the ones called upon when certainty 
of the outcome was more than desirable, but rather a necessity. Using 
these leaders as shining examples of what a good Naval leader should be, 
the Navy ingrained in its leaders the “tradition" of outstanding 
leadership, laced with names such as John Paul Jones, Truxtun, 
Bainbridge, Porter, Perry, Farragut, Taussig, Halsey and Nimitz. In 
order to be a really great leader one had only to emulate these fine 
examples, the Great Man concept. The traits that great leaders viewed as 
being desirable were obviously the traits that would create good leaders. 
Hence, John Paul Jones' thoughts in "Code of a Naval Officer", were 
memorized by midshipman as the proper conduct for a Naval officer. 
Pleble's emphasis on discipline and drilling was the gospel on the 
training of subordinates, while Farragut's disregard of "Torpedoes" was 
an example of leader courage. Additionally, what was left unsaid but 
amply demonstrated, was that good leaders would inevitably become famous 
for their abilities. Hence, an end means inversion occured developing 
the notion that, if one were well known or famous, then one must surely 
be a great leader. 
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Until World War II, this idea of a tradition bound leader was the 
most prevelant view held throughout the Navy. During WW II with the 
large influx of personnel due to a quickly expanding Navy, leaders not 
ingrained in tradition were introduced. This large influx of leaders at 
all levels could not be assimilated into this tradition bound leadership 
style. Success or failure of their personal leadership styles were based 
largely on the end results achieved, which may or may not have been 
steeped in tradition. Since on-the-job leadership training was not 
practical, it was viewed that some rudimentary training in leadership 
could avoid the pitfalls of this trail and error method of leadership. 
Leadership training would improve the performance of those born leaders 
as well as those less capable personnel occupying leadership positions. 

Since World War II, the Navy has emphasized its training at the be¬ 
ginning of a person's military service with such programs as boot camp. 
Officer Candidate School, and "Plebe" summer at the Naval Academy. 
Occupational training is generally given immediately after boot camp and 
prior to the assignment to his permanent unit. Contained within these 
programs were included some aspects on "leadership" training as well as 
"discipline". For the enlisted personnel the emphasis was on discipline, 
the necessity of promptly obeying lawful orders with adherence to 
organizational norms in the form of Naval regulations. For the officer 
the emphasis was on the basic responsibilities as a leader and the rudi¬ 
mentary skills needed to direct the efforts of others. Leadership con¬ 
tinued to remain a near sacred term, steeped in tradition and assumed to 
be something a person was born with or without. Until the 1950's, 
leadership training within the Navy was not viewed with any serious 
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concern for modification. New management ideas developed by the civilian 
sector were largely ignored and not incorporated within any Naval 
training. In a Navy where disciplinary standards were well established; 
manpower supplies were adequate to sustain manning levels; and training 
costs for necessary skills were not out of control, there was no perceiv¬ 
ed need for further leadership training. 

B. DEVELOPMENT OF NAVY LEADERSHIP TRAINING 
1. The 1950’s 

With the introduction of the Uniform Code of Military Justice 
[UCMJ] into the Navy in 1951, the nation's concern for the individual 
rights of soldiers and sailors became evident. This necessitated an 
examination of the traditional leadership style then practiced throughout 
the Navy. Soon this concern became focused on symptoms that were viewed 
as evidence of poor or misdirected leadership. Sources for this concern 
included: responses from two thirds of a sample of 10,000 U. S. Navy 
sailors who perceived that their officers and petty officers were not 
interested in them as human beings; a brig population that had grown 
equal in size to the entire U. S. submarine force; and, the shocking 
information that some 70 percent of the U. S. prisoners of war in Korea 
had actively collaborated with their captors.[HUBBELL,1960] 

Under this pressure and influence, line officers along with 
civilian leaders began to take a closer look at leadership training with¬ 
in the Navy. On May 17, 1958, General Order 21 was issued by the 
Secretary of the Navy to all commanding officers to integrate leadership 
training into the technical training of their men.[CNP,1963] The result 

of this order was a succession of haphazard training efforts that 
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lacked any clear purpose, theoretical basis, method or goal. Individual 
training programs reflected each commanding officer's own ideas, educa¬ 
tional background and knowledge of leadership, as well as how to best 
teach it. Regardless of the ineffectiveness of General Order 21, it did 
demonstrate a commitment to leadership training from the upper echelon of 
the Navy. However, changing emphasis in programs reflected a continuing 
frustration at not achieving the hoped for levels of productivity, con¬ 
formity and retention from its personnel. 

2. The 1960's 

General Order 21 did not meet with its intended success. The 
initial acclaim and status that the leadership training program gained as 
a separate division in the Bureau of Naval Personnel was lost, as hopes 
faded with a lack of tangible results. Reissuance of the order in 1963 
did nothing to alleviate the situation. In 1966, Naval leadership 
training was incorporated into the broader program of General Military 
Training [GMT].[PARKER,1980] Leadership training requirements in GMT 
were reduced from previous levels. Commanding officers were committed to 
train their enlisted personnel ten hours per year in subjects such as 
leadership style, the chain-of-command and authority, responsibility and 
accountability.[AUEL,1975] 

Individual commanding officers, still responsible for implement¬ 
ing the training, saw little evidence of clear purpose or benefit from 
the program. The leadership training was typically delegated to a less 
capable or already overburdened junior officer. The decline of this 
leadership program marked the last time operational commanding officers 
would be entrusted with any formal leadership training program. 
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Succeeding programs would be centrally controlled and conducted by the 
Navy training establishment. The decline and replacement of this program 
typifies the course of such efforts within the Navy. As one analysis 
suggests, "The leadership program fell victim to its own frills and was 
downgraded by Navy Institutionalists, because it was an [Secretary of the 
Navy] intervention without sufficient input from line managers." 
[AUEL,1975] 

3. The 1970* s 

The Navy in 1970 was in a state of rapid change. Society's 

problems had spilled over into the Navy while the Vietnam conflict had 

severely tarnished the military image. The traditional character and 

structure of the Navy was coming under question. Drug and alcohol abuse 

as well as racial problems were spreading. Imminent conversion to an 

all volunteer force promised only additional problems for Navy leaders. 

Amid this social upheaval and pressure for change, a relatively young and 

unconventional Chief of Naval Operations [CNO], ADM Elmo R. Zumwalt, Jr. 

assumed command. In his first year, only nine and one-half percent of 

eligible personnel re-enlisted, far below the nunber required to maintain 

mandated personnel strength levels.[PARKER,1980] The new recruits 

necessary to reduce this shortage would ensure that societal problems 

would continue to infiltrate and dominate the military scene. ADM 

Zumwalt expressed his leadership philosophy in the following terms: 

...The style of leadership that accorded best with my 
own inclinations and operational efficiency was one of 
treating subordinates with consideration and respect. I 
had not found that a 'tight' ship had to be an 'uptight' 
ship and I had hoped that sooner or later the Navy would 
give institutional recognition to this principle....by 
overhauling such of its procedures and jettisoning such 
of its traditions as encouraged martinetism and 


20 






martinets.I am certain that what finally decided 

Secretary of Defense, Melvin Laird, and the Secretary of 
the Navy, John Chaffe, to risk junping me into the posi. 
tion of Chief of Naval Operations over the heads of 
thirty.three of my seniors was my advocacy of rapid and 
drastic changes in the way the Navy treated its uniformed 
men and women.[ZUMWALT,1976,p.167-169] 

Using a group of hand picked senior officers in the Bureau of 
Naval Personnel, ADM Zumwalt tried to establish new philosophies and 
procedures. Intending to bring dignity, pride and self-esteem to 
sailors and to end the "Mickey Mouse" rules, ADM Zumwalt commenced re¬ 
formulating Navy personnel management policies with his infamous "Z. 
Grams".1 Training programs in the areas of drug and alcohol abuse, 
race relations, overseas diplomacy and leadership were established. The 
entire effort was labeled as the Human Goals Plan. 

One of the most controversial and opposed aspects of the Human 
Goals Plan was its top-down, OMBUDSMEN-like feature. For example. Head, 
quarter's "...program managers had wide latitude to intervene at any 
level in the Navy organization with stringent requirements for individ¬ 
ual ship participation..."[AUEL,1975] This feature was deeply resented 
by many senior Navy officers and petty officers. They claimed Human 
Goals mandated actions eroded discipline, took too long, ignored 
operating requirements, and resulted in a loss of the immediate superi¬ 
or's credibility.[AUEL,1975] Thus, ADM Zumwalt’s view of an urgently 
needed effort to rid the Navy of rigidity, conflicted with his oppo¬ 
nent's views of a breakdown in discipline in working outside of the 
chain of command. 

IZ-Gram was a phrase used to denote ADM Zumwalt's CNO policy 
memorandums. Sent simultaneously to all levels of command, this rapid 
distribution resembled the sending of a telegram, hence the phrase, 
Z-Gram. 
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Opposition to A0M Zumwalt's approach and its unworkability re¬ 
sulted in a structural change in late 1972. The program continued, with 
an effort to return it back under the chain-of-command. "The entire 
effort had grown quickly, contained many unqualified people in important 
jobs and lacked the necessary objectives and assessment machinery to de¬ 
monstrate what, if anything, was being accomplished."[PARKER,1980,p.7] 
However, responsibility and reporting arrangements were not the only 
problems. “The vague goals of bringing equity, developing management 
skills and improving personnel performance were well-intended, but un¬ 
coordinated, unevenly managed, and generally ineffective."[PARKER,1980, 
p.7] The problems were compounded by a lack of evaluation to de¬ 
termine its progress or overall success. 

The new program. Human Resources Management [HRM], represented 
more of a change in name and structure rather than content. Fleet 
commanders and other line managers were delegated the responsibility of 
continuing these programs. The leadership training program continued 
basically unaltered with the exception of a new name. Leadership and 
Management Training [LMT]. LMT consisted of a ten day course attended 
by personnel in the top four enlisted pay grades and the lower four 
officer pay grades. Like those programs that preceeded it, LMT lacked 
clear objectives, theoretical basis, and a plan for evaluating its 
effectiveness.[PARKER,1980] In addition, each school developed its 
own curriculum and style resulting in wide differences between schools 
as to what was actually being presented. 
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Racial incidents in 1972 onboard the aircraft carriers Kitty 
Hawk and Constellation , and the fleet oiler Hassayampa , created opposi¬ 
tion to ADM Zumwalt's programs from outside the Navy. Many placed the 
blame on "permissiveness" resulting from ADM Zumwalt's humanistic 
policies.[PARKER,1980] A special subcommittee of the House Armed 
Services Committee headed by Representative Floyd V. Hicks, in calling 
for greater emphasis on leadership training in the Navy, released the 
following statement: 

"One of the most alarming features of the investigations 
was the discovery of lack of leadership by middle manage¬ 
ment in the Navy. It became apparent that while junior 
officers, chief petty officers and senior petty officers 
were performing their technical duties in a proficient 
manner there was a lack of leadership in dealing with 
seamen."[HOUSE OF REPRESENTATIVES,1973] 

In mid-1974 ADM Zumwalt was replaced as CNO with ADM James R. 
Holloway, III, a more conventional leader in terms of Navy leadership. 
ADM Holloway was concerned with the seeming lack of discipline, lost 
pride, and absence of leadership especially within the ranks of the 
Navy's middle managers. As CNO, he set as one of his primary goals, the 
improvement of senior petty officer and junior officer leadership per¬ 
formance. As a result, on 28 August 1974 a study was undertaken to 
determine the cause of the lack of leadership. More importantly, the 
study would assess the needs of the Navy in terms of leadership training 
and design improvements necessary to upgrade LMT to meet these needs. 
This action would ultimatly lead to the development of the Navy's 
Leadership and Management Education and Training Program [LMET], but 

would not actually commence for another four years. 
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C. LEADERSHIP AND MANAGEMENT EDUCATION AND TRAINING [LMET] 

In early 1975 the Chief of Naval Operations [CNO], ADM James L. 
Holloway, III, and top Navy leaders found themselves in a dilemma. A 
comprehensive review and examination of existing leadership and manage¬ 
ment training had revealed 58 formal training courses and 11 correspond¬ 
ence courses costing 12.8 million dollars a year.[AUEL,1975] Neverthe¬ 
less, contemporary Navy problems such as high attrition, low retention, 
absenteeism, substance abuse and crisis management among its personnel 
were continuing to be sited as major problems. In an effort to rectify 
the situation, after internally generated Navy studies had produced no 
conclusive results, several civilian contractors were asked to submit 
training proposals and McBer and Company, a Boston-based research and 
development firm, was chosen to develop a program to address the Navy's 
leadership and disipline problems. The Navy continued with its broad 
based leadership training [LMT], with emphasis on general knowledge of 
management concepts and theories until McBer cou^d develop the proposed 
skill acquisition program of job relevant, specific practices and 
techniques. 

1. The Navy LMET Plan 

The Navy's plan under McBer would be to develop Naval personnel 
at each level of competence required by their jobs. In doing so, the 
new program's mission would be "to Increase the Navy's ability to 
achieve its overall mission by increasing the effectiveness of Navy 
leadership across all levels of the chain-of-command."[CNET,N.D.,p.5] 
The endeavor would be a systematic training program, researched based 
upon Navy situations aimed at specific billet levels, emphasizing 
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individual initiative and accountability. Specifically, its goals would 
be: 


To provide a formal and systematic training program for 
professional development of Navy leaders at critical 
points in their careers, based on research of effective 
Navy leadership. 

To train officers and petty officers in the specific 
leadership and management skills needed to perform 
effectively at their level in the chain-of-command. 

To conduct ongoing evaluation for improving and updating 
these programs. 

To encourage Navy leaders to take personal responsibility 
for implementing effective leadership skills, by means of 
an educational approach that emphasizes individual initi¬ 
ative and accountability for effective performance as a 
Navy leader. [HRMC,N.0.,p.7] 

The formal Navy training plan was issued in February 1979, 

several months after the commencement of LMET course training. That 

plan addressed the results expected: 

Improved leadership and management competence on the 
part of the Navy's officers, petty officers and civilian 
personnel will enhance the Navy's performance in all 
mission areas, and may well provide the margin of supe¬ 
riority at sea that the Navy can achieve over any 
potential future adversary. It will also aid in the 
resolution of contemporary Navy problems involving 
retention, crisis management, disciplinary rates, 
attrition, working conditions, etc. LMET is designed to 
improve and maintain the requisite level of leadership 
and management competence through the Navy total force. 
[CN0.1979] 

2. Theoretical Bases for LMET 


Leadership and Management Education and Training [LMET], the 
Navy's sole integrated approach to the leadership training problem was 
developed by McBer and Company. McBer and Company was set up in 1970 by 
Harvard professor David C. McClelland and psychologist David Berlew - 
hence the name McBer. 
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McClei land,an eminent clinical psychologist, is well known for 
his research on power and achievement motives.[G0LEMAN,1981,p.35] In 
his book. The Achieving Society , he discusses research which sought to 
isolate certain psychological factors, particulary the need for achieve¬ 
ment and to assess their impact upon the economic growth and decline of 
societies. Using scientific, quantitative methods, coupled with the 
psychological knowledge of human motivation, he demonstrated how the need 
to achieve motive influenced economic development, which, more important¬ 
ly, could better equip man to shape his own destiny.[MCCLELLAND,1961] 

Convinced of the achievement motive, McClelland undertook re¬ 
search in the area of motive acquisition. Both behavior theory and 
psychoanalysis agree that stable personality characteristics, like 
motives, are laid down in childhood and difficult to change. 
McClelland's attempt to somehow change these motives in adults raised 
problems of both an ethical as well as a methodological nature. Clearly 
there were processes in the past which had successfully altered 
personality change - most notably devout missionaries.[MCCLELLAND,1965, 
p.322] Using empirical information from behavioral learning experi¬ 
ments, a strategy was set up in an attempt to change or promote the 
achievement motive, thereby improving entrepreneurial performance of 
businessmen. 

McClelland's motive development program was centered around 
twelve propositions of motive acquisition, notions mostly backed by 
empirical information or research from various studies. The training 
program was designed for small groups [under 25] to be taken voluntarily 
over a short duration of time [1-3 weeks]. The educational design of the 
course used various methods of content and process presentations to 

26 






achieve the overall goal. The course normally ended with each partici¬ 
pant preparing a written docunent outlining his goals and life plans for 
the next two years. McClelland states, “The participants were to regard 
themselves as "in training" for those next two years, in that 10-14 days 
is obviously too short a time to do more than conceive a new way of life: 
it represents the residential portion of the training only." During 
these two years, questionnaires would be sent out each 6 months to remind 
them of the issues discussed and to give them information to help 
determine how well they were progressing on achieving their self made 
goals.[M cClelland, 1965,p.329] 

After repeatedly giving the course both at home and abroad, 
McClelland's data seemed to suggest that one third of the people remained 
relatively unaffected while the other two thirds represented a doubling 
of the normal rate of unusual entrepreneurial activity.[MCCLELLAND,1965] 

McClelland's propositions were stated generally enough so that 
other terms such as "attitude" or “personality characteristics" could be 
substituted for the term "motive". In this way he believed the proposi¬ 
tions would also hold true for other personality variables.[MCCLELLAND, 
1965,p.3323 

As a final note on his motive development program of training 
McClelland wrote, "...rather than developing “all purpose" treatments, 
good for any person and any purpose, it [psychotherapy] should aim to 
develop specific treatments or educational programs built on laboriously 
accumulated detailed knowledge of the characteristic to be changed." 
[MCCLELLAND,1965,p.333] McClelland's motive development program was to 
be a significant building block in what would become the Navy’s LMET 
Program. 
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In early 1971, McClelland [MCCLELLAND,1973], criticized the 
intelligence or aptitude testing movement in a public lecture given at 
the Educational Testing Service in Princeton, New Jersey. McClelland 
criticized the intelligence tests just as he had done 15 years earlier, 
on the grounds that while they may predict good grades in school they 
have little effect in predicting good life outcomes. His own research 
supports this notion. Good test scores allow people to get into better 
schools, however, after graduation the significance ends - good test 
score and bad test score graduates both succeed and fail. McClelland 
argues that a much wider array of talents or compentencies should be 
assessed to determine college entrance, with an emphasis on "grades in 
life" and not "grades in school".[MCCLELLAND,1973,p.7] Good grades in 
school falsely lead people to believe that they are more competent and 
therefore more likely to do well in life. 

McClelland listed six principles as paramount in his alternative 
approach, compentency testing, to traditional intelligence testing. Of 
the six, two are particularly significant in understanding overall LMET 
development. 

The first principle concerns criterion sampling. Criterion 
sampling means observing people in the field and analyzing their perfor. 
mance. A test that is directed at the components of this performance 
would be the best test. McClelland states, "If you want to test who will 
be a good policeman, go find out what a policeman does. Follow him 
around, make a list of his activities, and sample from that list in 
screening applicants."[MCCLELLAND,1973,p.7] General intelligence and 

the ability to play word games may not be the best test in determining 







who to hire as a policeman - a movement toward behavioral analysis is 
required. 

The second r-inciple concerns the ability to assess competencies 
involved in clusters of life outcomes. While criterion sampling can be 
used to identify occupational competencies unique to a special job or 
task it is also necessary to identify social competencies that contribute 
to successful performance. McClellano writes that, "Some of these 
competencies may be traditional cognitive ones involving reading and 
writing. Others should involve what traditionally have been called 
personality variables, although they might better be considered 
competencies."[MCCLELLAND,1973,p.10] McClelland gives four illustra. 

tions of these social competencies, each with supportive research for 
both its significance as well as insights into its measurement, they are; 
communication skills, patience, moderate goal setting, and ego 
development.[MCCLELLAND,1973] 

3. LMET Study for the Navy 

Using McClelland's research and conceptual ideas on job compe. 
tency, McBer and Company, working in close cooperation with the Navy's 
Bureau of Personnel [8UPERS], began developing LMET in 1976. Similar 
methods and techniques used during the past decade in civilian organi. 
zations were adapted for the Navy's overall LMET design. The first task 
was to identify what Navy leaders, particularly superior Navy leaders 
actually do in handling leadership and management tasks. As Dr. D. G. 
Winter explains: 

The traditional way of answering the question, "What 
makes a good Navy leader?", is to ask people what 
personal qualities and skills they believe are important 
for leadership and management in a particular job. 








Putting together the answers of people actually In the 
job, the answers of superiors [or supervisors], and the 
answers of their subordinates [or clients] ought to give 
us a collective theory of what that job requires. In 
fact, however, this procedure is likely to give us a 
mixture of highly specific job tasks and vague plati. 
tudes about personal qualities, while neglecting the 
qualities that are crucial for excellent or superior 
performance.[WINTER,1979,p.2] 

McBer believes that it is more accurate and relevant to find 
out who are the superior and average leaders from the personnel working 
closest with them, and then identify what distinguishes them from each 
other. 

Using Navy commands in San Diego, California and Norfolk, 
Virginia, McBer asked commanding officers to identify superior and 
average leaders at eight career points: division officer, department 
head, executive officer, and commanding officer for commissioned 
officers, and petty officer, leading petty officer, leading chief petty 
officer, and master chief petty officer of the command for non.commis, 
sioned officers. In this way, 51 people [30 superior, 21 average] were 
identifed from the Pacific Fleet and 78 people [38 superior, 40 average] 
from the Atlantic Fleet. Most combinations of warfare community and 
career points were represented in this total sample of 129 officers and 
enlisted.[WINTER,1979] 

The personnel were interviewed individually and asked to relate 
three incidents in which they/felt they did very well, and then three 
incidents in which they did not feel very successful in their present 
jobs.2 The interviews were recorded almost verbatim and analyzed 

?The McBer approach is not unique in all respects. The 
critical - incident interview was developed by psychologist John 
Flanagan during WW II.]G0LEMAN,1981,p.39] McBer calls his data 
gathering technique behavioral event interviews [BEI]. 
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carefully to determine what similarities superior Naval leaders had shown 
that average leaders failed to show. The interviews of 36 of the Pacific 
Fleet sample were used to create a series of 27 competency elements, 
grouped into 5 clusters. When scored on the whole sample of 51 Pacific 
Fleet interviews, most of the 27 competency elements differentiated the 2 
groups.[WINTER,1979] 

To validate their findings, McBer first used the Atlantic Fleet 
sample scored by interviewers who did not know if the person was 
"superior" or "average". Additionally, a second validation technique was 
used involving a new, much larger sample. An extensive battery of paper 
and pencil tests was developed and administered to over 1,000 Navy 
personnel from petty officer through commanding officer, in the 3 warfare 
communities from both fleets. Interviews which could be scored for 

competency elements were available on 61 of these persons.[WINTER,1979] 
Further analysis was conducted using 220 persons from the larger sample 
size predominately in determining variations of leadership and management 
skills on a hierarchial level. 

Once a competency element derived from the original interviews 
with the sample of Pacific Fleet personnel had been validated by either 
of those two procedures, it was considered to be a competency that is 
associated with superior leadership and management performance in the 
Navy. Sixteen of the original 27 elements were validated in this way.3 
[WINTER,1979] Those 27 initially identified elements served as the basis 
for original LMET curriculun design.[PARKER,1980,p.15] 

^For a more indepth description of competency identification and 
validation the reader should consult the Winter Report [WINTER,1979] on 
LMET theory and research. 
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These 16 validated competencies are at the core of the LMET 
Program. The basic premise of LMET is that increases in any or all of 
those competencies will be associated with improved leadership 
performance in the Navy.[WINTER,1979] Today, the 27 competency 
elements have subsequently been reduced to 16 competencies which are 
grouped into 5 competency clusters for training purposes.* 

4. The LMET Competencies 

A competency is any knowledge, skill, attitude or value which 
can be shown to distinguish reliably between effective and less 
effective job performance. As Richard Boyatzis, President of McBer and 
Company explains, "Competencies are not aspects of a job, but rather 
characteristics of the people who do the job best."[GOLEMAN,1981,p.40] 
In other words, a competency is what superior performers do more often, 
in more situations, for better results than do average performers. 

The first competency cluster is a concern for efficiency and 
effectiveness, paraphrased as "doing things well, and wanting to do 
better."[HRMC,N.D,p.11A] Major components of this group include 
setting goals and performance standards, and taking initiative, with 
the concern for achievement as an underlying thought. 

The second competency cluster is the skillful use of influence 
or "using influence in a positive fashion...not as a personal end « but 
toward Navy goals and effectiveness."[HRMC,N.D.p.l2A] Major components 
include Influences, team building, development of subordinates and 
self-control, with the concern for influence as an underlying thought. 


*A complete list of the 5 competency clusters is contained in 
Table 1. 
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TABLE 1 

LMET COMPETENCY CLUSTERS 


1. Efficiency and Effectiveness - “Doing things well, and wanting to do 

better." Components include: 

* Setting goals and performance standards 

* Taking initiative 

2. Skillful Use of Influence - "Using influence in a positive fashion,.. 

not as a personal end but toward Navy goals and effectiveness..." 
Components include: 

* Influences 

* Team builds 

* Develops subordinates [coaches] 

* Self,control 

3. Advising and Counseling . "Advise and counsel personnel in order to 

improve their performance on the job." Components include: 

* Positive expectations 

* Realistic expectations 

* Understanding 

4. Management Control - "Optimizing people and resources to the task." 

Components include: 

* Plans and organizes 

* Optimizes use of resources 

* Delegates 

* Moniters results 

* Rewards 

* Disciplines 

5. Conceptual Thinking , "Applies concepts to a job situation." 

Components include: 

* Conceptualizes 


Reference: U. S. Navy Human Resource Management Center, LMET 
Overview Brief, no date. 












The third competency cluster is that of advising and 
counseling, further amplified as to "advise and counsel personnel in 
order to improve their performance on the job." [HRMC.N.D.,13A] Major 
components of this group include positive expectations, realistic ex¬ 
pectations and understanding. 

The fourth competency cluster is that of management control or 
to "optimize people and resources to the task." [HRMC.N.D.,14A] Major 
compontents are plans and organizes, optimizes use of resources, 
delegates, monitors results, rewards and disciplines. 

The final competency cluster is that of conceptual thinking or 
"identifying and organizing relevant facts to gain a clear understand¬ 
ing of the situation before acting on it." [HRMC.N.D.,15] The only 
component of this cluster is that of applying concepts to a job 
situation. 

In addition to the five competency clusters, the McBer study 
identified six basic categories of leadership styles used by the 
officers and petty officers interviewed, they are: 

a. Coercer - all stick and no carrot 

b. Authoritarian - firm but fair 

c. Affiliator - people first, task second 

d. Democrat - participative manager 

e. Pace Setter - do it myself 

f. Coach - Management by Objectives [MBO] Manager 

McBer found that superior leaders tend to be more skilled in several 
styles, recognizing which style fits a given situation. LMET en¬ 
courages situational leadership with the development of a larger re¬ 
pertoire of leadership styles.[HRMC,N.D.,p.16] 
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5. LMET Structure 

LMET Is now the formally approved model for Navy leadership 
training, although it is not yet completely implemented. To achieve 
standardized implementation of the program throughout the Navy, the 
Chief of Naval Education and Training [CNET], has been designated 
training agent. 

The first LMET classes opened in 1978, and the program has 
expanded rapidly. Projected yearly outputs for fiscal years 80-84 are 
4,840 officers and 12,242 enlisted based upon the nunber of school 
quotas available.[HRMC,N.D.] There are currently five levels of LMET 
with special emphasis on fleet personnel attending first. These levels 
include; commanding officer/executive officer, department head, 
division officer, chief petty officer and petty officer. Officer LMET 
is further subdivided with separate classes for aviation, submarine, 
and surface ship officers.[HRMC,N.D.] Personnel returning to the 
fleet normally receive LMET enroute to their new permanent duty 
station. School quotas can also be obtained by Navy commands for 
personnel already at their command on a temporary duty basis. However, 
these quotas are extremely difficult to obtain and are offered only on 
a space available basis. 

LMET is now taught at existing Navy training centers. Sites on 
the east coast include: Memphis, Tennessee; Little Creek, Virginia; New 
London, Connecticut; Mayport, Florida; Pensacola, Florida; Newport, 
Rhode Island; and Charleston, South Carolina. Sites on the west coast 
are: Coronado, California; San Diego, California; Bangor, Washington; 
Treasure Island, California; and Pearl Harbor, Hawaii. 
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6. LMET Instruction 


All classes last 10 working days, except for the 12 week LMET 
instructor course taught at Memphis, Tennessee. Instructors are deemed 
to be the key to success of the program. Accordingly, instructors have 
been selected against rigorous criteria and trained in LMET concepts 
and methods by the program designers. No one is allowed to teach LMET 
who is not a graduate of the 12 week LMET instructor course .[0LMSTEA0, 
1980] 

LMET class sessions are team-taught by two or three instructors, 
with officers teaching officer courses and senior petty officers teach¬ 
ing equivalent enlisted personnel. Reportedly, classes generally consist 
of 20-24 students. 

Each LMET course is based around the competency notion and the 
five competency clusters. The training sequence for each competency 
consists of lectures to identify and assess how the competency applies 
in the Navy. Through self-analysis exercises and pretesting, students 
discover how they learn, as well as the type of management styles they 
possess. Skill acquision and practice are developed through games, 
role-play exercises and case studies. Finally, through various pro¬ 
cesses the students learn to apply the competencies to tasks similar to 
those which will be encountered in their new job assignment. 

The personal comprehensive plan is the final activity in 
all LMET courses. It is a statement of personal goals, 
shaped to the concepts and language of the leadership 
competency model. Participants are encouraged to formulate 
and describe realistic yet challenging goals that are appro¬ 
priate to their Navy leadership situations, to become aware 
of difficulties and obstacles to these goals, and to nrlte 
out specific action steps to overcome the obstacles and 
attain the goals. 
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A distinctive feature of LMET courses is the personal student 
log, which the participant retains as a written record of the 
LMET experience. It contains his or her self-assessment, life 
situation, and goals - all phrased in terms of the competency 
language.[WINTER,1979,p.11-12] 

Differences [PARKER,1980] between classes for higher or 
lower ranking personnel are found primarily in the content, setting, and 
cast of characters used in the role-play situations, case studies and 
general class discussions. Chief petty officer and petty officer 
courses have the same level of emphasis, differing primarily in the 
seniority of instructions that teach the two levels. 

The LMET classes are reportedly [PARKER,1980] well-paced, 
shifting from lecture to discussion to small group activities in a 
fashion intended to maintain student interest, to provide frequent 
opportunities for students to express feelings and opinions in a 
supportive environment, and to trade ideas with their peers. In actual 
practice LMET courses may differ slightly from location to location and 
class to class as instructors seek to motivate and influence each 
particular group. Student critiques completed at the end of the course 
indicate that students like LMET.[PARKER,1980] 

7. Future for LMET 

As mentioned earlier, LMET has not yet been completely 
implemented. Planning through fiscal year 84 will include courses for 
more junior petty officers as well as shore commands. Navy civilians, 
and Naval Reserve personnel, with the overall goal of training everyone 
in a Navy leadership position. The cost of LMET during 1980 was 17 
million dollars; for 1982-86, the projected cost is slightly more than 
29 million dollars per year.[PARKER,1980] 











Curriculun modification, concentrated at the LMET Instructor 
School in Memphis, has gone on almost continously since the problem 
began. Efforts to improve the program are concentrated in Memphis under 
the direction of CNET. Over and above internal course improvement and 
development, however, LMET has three longer-range goals: 

a. Additional research is planned to provide further vali¬ 
dation and cross-validation of the competencies themselves. Of 
particular concern here is pinpointing in greater detail the re. 
lative importance [and hence the appropriate instructional 
weighting] of each competency at a few "key" Navy billets - for 
example, at the commanding officer level and at the petty 

officer level. 

b. The test battery will be revised and expanded, in order 
to provide instruments and procedures that can be used to assess 
individual standards of performance against the competency 
profile of superior Navy leaders. When available, such instru¬ 
ments could also be used before and after courses, as measures 
of the immediate effectiveness of a person's participation in 
LMET. 


c. The LMET program will be evaluated. Short-term, inter¬ 
nal course evaluation will provide an immediate indication of 
the effects of LMET. Most important, however, is the more 
difficult long-term external evaluation. Do LMET graduates 
perform at significantly higher levels of leasership than non. 
graduates? Does the LMET progren have a measurable impact on 
standard Navy indicators of leadership and management perform¬ 
ance, as well as on newer indicators that will be developed as a 
part of LMET itself? [WINTER,1979,p.12-13] 


D. NEED FOR LMET EVALUATION 

LMET will soon begin its fourth year with graduates continuing to be 
dispersed throughout the entire Navy community, limited only by cost con- 
serations and the shear nunbers of quotas available annually. Plans to 
continue LMET implementation on a more widespread basis are ongoing with 
the full backing of the CNO. The implication is that LMET has become a 
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solution, in great part, to the Navy's overall leadership and discipline 
problem. Most people feel good about the direction of LMET and its 
results, however, elaborations are few and comments are less than 
specific. 

Just how good is LMET? Is LMET producing the desired results or 
changes? Unfortunately, those questions remain unanswered. It is diffi¬ 
cult to identify what is meant by "improving" effectiveness. Efficiency 
can usually be measured in terms of cost and time. Defining effective¬ 
ness requires a detailed understanding of the variables that affect 
performance. 

Several prominent industrial psychologists express doubt 
about the prospect of meaningful evaluations of methods like 
McBer's. It is simply too difficult to identify the qualities 
of good managers, they say, let alone measure their validity as 
predictors of performance. [G0LEMAN,1981,p.46] 

Neither McBer nor its clients have so far produced much empirical 
proof that their method does, in fact, lead to demonstrable improvements 
in job performance. There are few published studies of the effectiveness 
of comptenecy-based selection and training as practiced by McBer. 
[G0LEMANJ981] 

Much of McBer's work is too recent for meaningful evaluation. Ex¬ 
cept for end of course critiques by students and informal comments by 
commanding officers, LMET falls into this category. 

However, the honeymoon for LMET is beginning to end. Researchers 
are starting to question the McBer approach as unvalidated theory, ignor¬ 
ing a host of pet factors, such as situational leadership or content 

t 

verses process leadership, to name only two. Some claim the data gather¬ 
ing and analysis are not particularly impressive. Others insist that 
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the sample size to base this magnitude of a program on was just too 
small. [PARKER,1980] 

Additionally, there exists a very real need for all levels of Navy 
managers to know the degree to which the LMET Program is effective. 
Optimum use of scarce resources is a major concern to all managers, and 
the LMET Program continues to make demands for both manpower and money 
without proven justification. This concern should question whether the 
ten days at LMET school is in fact improving individual effectiveness as 
a leader, and hence worthwhile. If LMET is only a nice-to-know 
management course, producing few measurable results, then there would be 
a strong argument that those resources could be better allocated. On the 
other hand, if LMET graduates are more effective leaders and managers in 
directing the manpower resources available to them, then the program 
should be valued. The bottom line is that Navy managers have a real need 
for an LMET program evaluation to determine if the current level of 
support is warranted. 

An evaluation would also provide LMET program managers with feedback 
as to its effectiveness. It would identify areas that require further 
emphasis as well as identify deficiencies if they exist. The fact that 
continued evaluation of the program is a stated goal [WINTER,1979] 

further highlights this need. 

Perhaps more importantly, an evaluation can provide future direction 
for the program. This aspect can keep the LMET program ahead of, or at 
least, on top of emerging problems, thereby assuring it remains viable 
and healthy. With this forward looking attitude the program can become 
one of action vice reaction to the needs of the Navy. 
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The list continues on and so does the need for LMET evaluation. As 
with any program in times of austere funding, LMET will sooner or later 
come under careful scrutiny as to its cost effectiveness. The Naval 
Audit Service has already expressed an interest in examining LMET re¬ 
sults.[PARKER,1980] The Navy can little afford not to evaluate the 
program. 

As discussed at length above, the need for an evaluation of the LMET 
program is quite clear. Yet, the question of how to conduct this evalua¬ 
tion remains. How to best evaluate a system-wide program as extensive as 
LMET within the Navy is not readily apparent. The number of proposed 
methods would likely equal the number of people querried. System-wide 
surveys, periodic graduate questionnaries, unit reports on overall 
readiness vs number of graduates onboard, spot interviews, etc., are all 
possible approaches to evaluation complete with strong arguments both for 
and against their use. 

In deciding on the issue of how best to evaluate, one should 
consider what is being evaluated. LMET, as well as any program's 
effectiveness, is not easily measured. There are few clear cut effect¬ 
iveness indicators, no impeccable standards, and no completely reliable 
method of measuring effectiveness as has been discussed earlier. 

Another concern is the cost of any evaluation in terms of time, 
manpower and money. All three constraints further confuse the issue of 
how to best evaluate the LMET Program while optimizing these resources. 

This research is intended to provide program managers and decision 
makers with a practical method to answer the very question of how. This 
method of initial evaluation, in the form of a Pilot Study can test 
hypotheses, evaluate a methodology, uncover relevant issues while 
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providing data concerning the actual evaluation of a major program. This 
thesis will attempt to demonstrate the value and usefulness of the Pilot 
Study concept as related to a preliminary program evaluation. 


It is under this context that this research on IMET effectiveness 
was undertaken. The thrust of the work is twofold, first to try to shed 
some light on LMET effectiveness. Second and possibly more important, to 
demonstrate the practical value of a pilot study to help identify the 
relevant issues, evaluate methodology, and test hypotheses prior to 
committing extensive resources to a full scale evaluation. 
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III. CONSIDERATIONS IN DESIGN OF AN EVALUATION OF LMET: 
-GUIDHIHEST FOR T FTIOT STUDY- 


A. RATIONALE FOR A PILOT STUDY 

With the need for an evaluation firmly established, there remains at 
least two critical questions to be answered; who is to conduct the study 
and how? Let us briefly examine the who question first. There is 
logical support for McBer and Company to evaluate the LMET Program, since 
they are intimately familiar with its designed structure and theoretical 
basis. However, there exists a possible conflict of interest whereby the 
credibility of results could be in jeopardy. This in essence would be a 
form of self-evaluation, open to criticism as being favorably biased, 
regardless how fair or objective it was in reality. 

CNET is another possible choice for conducting the evaluation. CNET 
has familiarity with course content, teaching techniques, as well as easy 
access to graduates both before and after the training. Again for many 
similar reasons, the outcome of such an evaluation might be suspect. As 
part of the Navy bureaucracy, CNET would be placed in an untenable 
situation of evaluating a program that it administers. Any such 
evaluation would probably be subject to close scrutiny, reguardless of 
the actual quality. Clearly, there is a need for an independent 
evaluation of the effectiveness of this program to insure a relatively 
unbiased outcome. 

This leads to a third alternative, that of an independent outside 
contractor with no vested Interest in the outcome. Yet even this alter¬ 
native has considerable drawbacks. The cost of an evaluation would be 
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high, especially for a contractor who is unfamiliar with the program. It 
would be necessary for the contractor to expend a considerable amount of 
time, manpower and money in studying the program prior to commencing the 
evaluation. Even after that initial familiarization phase, there exists 
a very real possibility that the program could be misinterpreted, 
resulting in an inappropriate direction for data collection. This could 
result in an evaluation that uncovers nothing meaningful, or worse, mis- 
evaluates the program, causing additional expenditures to correct non- 
existant problems as well as creating the need for another evaluation. 

The need for an evaluation by a knowledgable but independent third 
party having no career-crucial or monetary interest in the outcome points 
to yet another alternate approach. An independent contractor working 
with only a sketchy concept of the program and little inside knowledge 
may not be a good solution. However, if that same contractor were pro¬ 
vided with some meaningful and concise direction for the evaluation, then 
the study could be conducted more efficiently with the necessary credi¬ 
bility. One method to provide this guidance is through a pilot study. 
The pilot study, conducted by personnel knowledgable of the organization 
and the program being evaluated, could identify issues, develop hypo¬ 
theses, and even test an evaluation methodology. By conducting a pilot 
study prior to the full-scale evaluation, information could be uncovered 
that would provide the program manager and/or contractor with the needed 
direction and relavent issues to effectively conduct a valid evaluation. 
Manpower needs for such a pilot study are minimal with a variety of 
possible sources such as CNET, HRMC's or, as in this case. Naval Post¬ 
graduate students working on their master's degree thesis. 
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It was with these thoughts in mind that a pilot study of the Navy's 
LMET Program was undertaken by two Naval Postgraduate students. The 
study commenced in mid-December 1980 and was completed in mid-June 1981, 
a period of approximately six months. 

B. EVALUATION DESIGN CONSIDERATIONS 

Having decided who should conduct the pilot study, the remaining 
question was how to evaluate the program and from there design the pilot 
study. The ideal design would have been to establish a control group of 
non-LMET graduates and compare their leadership/management practices with 
LMET graduates to see if there were any recognizable differences. Then, 
compare the non-LMET group with the LMET group to determine which command 
was in fact, best managing those problem areas which LMET was intended to 
alleviate. These would include areas such as drug and alcohol abuse, 
retention, crew morale, unit readiness and material condition, and 
command climate. Such an approach to evaluating a program would be very 
scientific and lend itself in establishing the effectiveness of a parti¬ 
cular program. The problem with implementing this approach is twofold. 
First, no such control group was established when LMET was started. As a 
result every command in the Navy has some LMET graduates on board. 
Secondly, there is the problem of determining if a command's management 
of those problem areas was in fact due to LMET training, some other 
factor, or more likely, a combination of more than one factor. 

Lack of a control group does not mean that a useful evaluation can 
not be accomplished. It does, however, lend support to conducting a 
pilot study in order to test an evaluation method prior to committing 
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scarce resources to a full scale evaluation. If unable to compare two 
groups such as, LMET and non-LMET graduates, then the next logical step 
is to compare individual performance before and after LMET training. By 
collecting behavioral data on managers before LMET school and comparing 
it to behavioral data gathered sometime after LMET school, changes in 
leadership/managerial behavior could be detected. The implication is 
that if the desired change is evident then the LMET training, hence the 
program can be surmized as being effective. There is at least one con¬ 
cern about this approach. That is, the LMET graduate needs to have had 
sufficient time after completion of the training to have developed a 
leadership/management behavior pattern. If conducted too soon after 
graduation the result may well be a parroting of LMET training with no 
real demonstrable behavioral changes. This need for a delay or time gap 
between data gathering necessitates an evaluation that could be quite 
lengthy in time. 

While this time gap may suffice for a full scale evaluation, it 
would, if used for a pilot study, create problems. By extending the 
length of time to conduct a pilot study its usefulness to a program 
manager is diminished. In a dynamic program such as LMET, an evaluation 
needs to be as timely as possible in providing information. If the pilot 
study is overly lengthy, the insights provided concerning program evalu¬ 
ation may no longer be applicable when presented to the program manager. 

If due to insufficient time, a before and after data gathering 

approach is impractical, then another method must be employed, that of 

the follow-up or post-test data collection. Using this method the intent 

Is to gather behavioral data after completion of the school to determine 

if the students behavior matches that which the program is designed to 
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produce. This follow-up or post-test method while the least rigorous of 
those mentioned lends itself quite well to a pilot study. It can provide 
sufficient data to identify issues, test hypotheses and evaluate a meth¬ 
odology for conducting a full-scale evaluation. In addition, this 
approach is not time constrained, is extremely flexible and easily adapt¬ 
ed to the resources required for a pilot study. A drawback to this 
approach is that data gathered from a student in a program, such as LMET, 
may be highly opinionated and not necessarily indicative of actual 
behavior, in essence only a self-report. This problem can be overcome by 
validating the subjects' self-reported data with additional information 
gathered from persons who have the opportunity to closely observe the 
subjects' behaviors. Supervisors, subordinates and peers are all 
potential sources of this validating data. 

One additional consideration deals with sample size. A full-scale 
evaluation in order to be considered valid needs to be a random sample of 
the population. Such a random sample can then be assumed to statisti¬ 
cally represent the whole population. When considering a major program, 
such as LMET, this random sample may need to be quite large. A pilot 
study, on the other hand, is not intended to be a substitute for a full- 
scale evaluation and need not use a random sample inorder to be useful. 
A sample that includes a cross section of affected people from the 
program is all that is necessary. The intent is to not bias the pilot 
study re- suits with data that applies to only a small segment or portion 
of the program population. The sample size for a pilot study, when 
possible, should provide a broad snapshot covering the entire spectrum of 
the program under evaluation. 
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C. GUIDELINES FOR AN LMET PILOT STUDY 

The need for an evaluation of the LMET Program has been established. 
Furthermore, the arguments in favor of conducting a pilot study in order 
to provide guidance, as to how to conduct that evaluation have been pre¬ 
sented. The remaining action is to formulate the guidelines necessary to 
conduct the LMET Program pilot study. 

The first concern was who should be included in the sample? 
Obviously, in order to conduct this LMET pilot study, information needs 
to be gathered on LMET school graduates as to behaviors they exhibit 
after the LMET training. This then identified the study sample, LMET 
gradu- ates. However, any data collected on the graduates which came 
from the graduates themselves would need to be validated as discussed 
earlier. This need for validating data also directed emphasis towards an 
LMET graduates Immediate supervisor and subordinate. The basic data 
package would be gathered from LMET graduates, with validating data 
gathered from their immediate supervisors and subordinates. 

Ideally, a graduate who attended LMET school after arriving at his 
present command would potentially provide the best behavioral change 
data. Unfortunately graduates in this category were expected to be 
difficult to find due to the programs scheduling. This scheduling 
established the normal routine of attending LMET school via Permanent 
Change of Station orders [PCS], that is before arriving at their command 
and not via returnable-quota. Temporary Additional Duty orders [TAD]. As 
a result of this procedure, most graduates would be expected to have 
attended LMET prior to reporting to a command, hence eliminating the 
"before" data. 
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Another issue in this question of who to interview was, which 
graduates? The LMET Program covers eight general management levels, they 
are: CO/XO, Department Head, Division Officer, Master Chief Petty 

Officer of the Command, Leading Chief Petty Officer, Chief Petty Officer, 
Leading Petty Officer and Petty Officer. These levels could be further 
grouped into CO/XO, Department Head, Division Officer, Chief Petty 
Officer and First Class Petty Officer which follows the normal Navy chain 
of command. Among these groups the lower 3 levels of Division Officers, 
Chief Petty Officers and First Class Petty Officers are the most numerous 
aboard any command, hence most promising as to accessability, thereby 
further defining the pilot study sample. 

Limited interviewer resources as well as time constraints limited 
the chance of obtaining what could even be remotely labeled a representa¬ 
tive sample. However, it needs to be emphasized that this was not to be 
an actual evaluation, but rather a pilot study. Therefore a non-random, 
cross section of these LMET graduates would be sufficient. In this way 
it would serve to evaluate methodology, identify key issues as well as 
provide some crude insights into LMET effectiveness. What now became 
important would be sampling a sufficient number of different commands as 
well as a variety of billets within those commands. Since operational 
commands had been given number one priority [0LMSTEAD,1980] for LMET 
school quotas it was possible to further restrict the sample to these 
commands. The most likely evidence of change would be among those 
commands which had the highest degree of exposure to the LMET Program. 

Due to the physical location of the Naval Postgraduate School on the 

west coast, a practical consideration was qiven to concentrate on west 

coast commands. Time was a major issue with travel costs providing an 
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additional incentive to remain on the west coast. With the standardi¬ 
zation of the LMET curriculum under CNET, it was felt that any general 
conclusions reached using west coast commands would most likely apply to 
east coast commands as well. With the geographic location narrowed down, 
type of commands on which to concentrate [operational i.e., air, surface, 
subsurface, amphibious, service force], and target billets identified 
[Division Officer, CPO, P01] the remaining question was how large of a 
sample would be necessary? The issue now became one of sampling a 
sufficient number of different operational commands [air, surface, 
submarine] as well as a variety of billets within those commands, within 
the limited time available. The aim was to minimize any biases peculiar 
to either type of command [surface, submarine, aviation]; billet 
position such as Division Officer, Leading Petty Officer or Chief Petty 
Officer; geographic location such as San Diego; or specialty area 
[weapons, engineering, supply, etc.]. This cross-sectional sample would 
be sufficient to draw conclusions and to provide certain guidance for a 
future full-scale evaluation. It is imperative that the limitations of 
this "non-random cross-sectional sample" be appreciated so as to prevent 
this research from being mistaken as a program evaluation. 

That leaves the remaining question of how to best gather the data. 
The choice of methods focused on two possible methods, interviewing or 
survey questionnaire. The collection of data, regardless of method, in¬ 
volves three elementary forms of human activity: observation, partici¬ 
pation and empathy. As best stated by R. L. Gorden: 








From these three activities all of the methods for study¬ 
ing hum an behavior are derived. Within this framework, inter¬ 
viewing is seen as one specific form of empathizing, partic¬ 
ipating and observing which takes place between two people... 
in this context the questionnaire [survey] is seen....simply 
as a technique for extending the interview in which the re¬ 
searcher participates by constructing the questionnaire. 
[G0RDEN,1980,p.5] 

Empathy is referring to feelings with another person or understand¬ 
ing how that person feels about something. Participation refers to doing 
something with another person in their regular ongoing activities. When 
taking on a participatory role one is in a vantage point from which it is 
possible to either observe another's activity or to introspectively note 
one's own thoughts and feelings as related to that situation. Observa¬ 
tion includes any sensory perception, not only visual, of cues which help 
a person understand human behavior. With these three concepts in mind 
let us examine these two data gathering methodologies. 

The first of the two methods is the survey questionnaire. Observer 
participation in a survey with the respondent can only be dealt with 
during the survey construction phase. Actual interaction between 
observer and respondent is extremely limited. During the construction 
phase it is possible to interject ones own thoughts as to what is 
important, what needs to be asked or which issues are relevant. Once 
constructed and administered, no further participation is available to 
the "observer". This inflexibility is a major weakness of a survey 
especially in a pilot study where the meaningful issues may not be known. 
Observation for a survey consists of looking at and examining responses 
written on the survey form. While perhaps easily quantifiable data may 
be gathered, data that can be easily analyzed, using computer programs, 
the observations are limited. The circumstance under which the answers 
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were written, such as, the respondents emotional level, reaction to the 
survey, workload and many other factors are lost to a survey administra¬ 
tor. There is also the concern of whether each item asked on the survey 
was fully understood by the respondent. Educational and cultural back¬ 
ground, socio-economic considerations, as well as the environment all 
have influence on the person participating in a survey. There is the 
argument that this prevents the observer from biasing the data. Yet in 
actuality the interviewers biases are permanently ingrained in the survey 
itself with no way of adjusting to different respondents short of re¬ 
writing the survey. By not viewing the respondent the interviewer looses 
that personal aspect of data collection. Any empathy must be transmitted 
via the survey instrument which is certainly difficult at best. 

All this is not too say that a survey does not have distinct advan¬ 
tages. A survey can be administered to a very large sample in a re¬ 
latively short period of time at a low cost. It also lends itself well 
to computer analysis. For data that is easily quantifiable into yes/no 
responses or to scales of like/dislike, a survey is especially appro¬ 
priate. However, when gathering data on a highly opinionated subject 
where there are no obvious choices or a potentially wide variety of re¬ 
sponses a survey can be useless. 

Another consideration is that survey construction is an exacting 
evolution. A good survey may take considerable time in constructing, 
testing and revising. The problem is once administered it is difficult 
to change so It must be correct the first time. It is this very in¬ 
flexibility that limits the usefulness of a survey in conducting a pilot 


52 








study, especially for a program where the real issues may not yet be 
known. 

The second method considered was the interviewing process. One 
obvious advantage to this method is the opportunity to gather data from 
multiple sources. Not only are there the verbal responses of the re¬ 
spondent but also non-verbal signals such as facial expressions, hand and 
body motions and tone of voice to name only three. All of these obser¬ 
vations can be used by the interviewer to gain a deeper insight into the 
responses to the interview questions. This multiple observation capa- 
bility can be enhanced by team interviewing, where one person questions 
and concentrates on verbal responses, maximizing the flow of data while 
the other person concentrates on note taking and non-verbal responses. 

Participation in the interview is another advantage over surveys. 
The observer is able to explore fully the responses given to questions by 
solicitating further clarification or in asking additional questions. 
This flexibility allows the interviewer to fully explore issues as they 
surface while ensuring that it is completely understood. This specific 
aspect fits the concept of a pilot study, especially the added flexi¬ 
bility of exploring new issues. 

Empathy can add to the quality of the data gathered via interview¬ 
ing. The interviewer in being able to develop empathy for the respondent 
during the course of an interview can anticipate probable reactions to 
questions and sense how the respondents feel about the events they are 
relating. This can add a qualitative aspect to what is actually being 
stated thereby further enriching the data. 
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All of this is not to say that interviewing does not have drawbacks. 
It is a time consuming process and costly in terms of money and manhours 
expended. There is a very real danger of the interviewers distorting the 
data with their own biases, perhaps unknowingly. Additionally, it takes 
a certain skill to establish a good rapport with the respondent. Without 
it, a free flow of information is not possible. 

The very flexibility of the interview poses another problem in that 
it *is very easy to get sidetracked into irrelevant issues or redundant 
detail to a response. The interviewing procedure is dependent on the 
free flow of information between respondent and the interviewer. 

The interview process itself can inhibit this very flow of inform¬ 
ation. Interviewing with two interviewers on one respondent may infact 
be intimidating to the respondent. If a tape recorder is used to record 
the verbal data it may similarly intimidate the respondent. 

An additional obstacle may be that of the actual person, officer or 
civilian, used in conducting the interview, especially for enlisted re¬ 
spondents discussing their supervisors. The Navy officer may have an 
advantage of being an insider to the organization, who understands the 
language and jargon and can probably empathize easily with the respon¬ 
dent. If in uniform he will present a familiar, possibly non¬ 
threatening appearance due to the respondents everyday interface with 
officers. A civilian on the other hand may hinder the flow of 
communication by having to have everyday situations and acronyms explain¬ 
ed. There is also the possibility of misunderstanding a word or phrase 
that is Navy jargon. 
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On the other hand the civilian interviewers may be less threatening 
in that they do not represent members of hiqher Navy management. For the 
Navy respondent, there may be less reluctance to voice a negative opinion 
about a Navy program to an outsider of the organization. 

0. SUMMARY 

These preceding considerations and issues concerning a pilot study 
for a major program, such as LMET, form the preliminary step in its 
overall evaluation process. While not all encompassing, each of these 
specific issues were addressed in formulating an LMET pilot study design. 
Any major program requiring evaluation could use this or a similar 
approach in order to develop an appropriate direction for a full scale 
evaluation. The pilot study design can provide decision makers with the 
relevant issues and guidance to better optimize their resources. 










IV. LMET PILOT STUDY DESIGN: A DESCRIPTION 


A. PREPARATION 

As in any research study, a pilot study being no exception, a plan 
of attack must be developed before one packs his bags and heads out to 
gather "data". The question of what one wants or seeks to find, should 
be clearly established from the start. Without this being established 
it is difficult, if not impossible, to best determine the course of 
action to take. Once the problem identification and statement is 
complete, a methodology and overall design strategy for the study can 
be formulated. 

An overall evaluation of LMET effectiveness would serve nicely as 
a problem statement for the proposed research. After all, the LMET 
Program had not been evaluated and effectiveness as related to im¬ 
proved student performance could provide a suitable criteria for 
measurement purposes. However, this was not to be the case for at 
least two major reasons, that of a time constraint and previously noted 
difficulties in measuring effectiveness. 

First, the two graduate students selected would have only 6 months 
to complete the study, from December through June. While sufficient 
time to commence an evaluation presented no major problems through June, 
both were scheduled to graduate at that time and would subsequently re¬ 
ceive seperate Navy orders based upon service needs. Since no other 
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graduate student assets were available with sufficient background 
experience and time to conduct the evaluation it was agreed to conduct 
a pilot level study rather than a major overall evaluation of LMET 
effectiveness. 

The second area of concern centered around the notion of effec¬ 
tiveness. Previous effectiveness studies, especially in the area of 
leadership training, in both the civilian and military sectors are few 
in nunber, yielding inconclusive results as a whole. It has been 
difficult at best to measure situational performance of personnel, let 
alone attempt to measure an improvement or increase in this performance 
and relate that to a specific training program to determine its overall 
effectiveness. The difficulty in determining effectiveness would only 
be compounded by the aforementioned insufficient time. The probability 
of obtaining an outcome would be low and any results obtained would 
lack credibility. 

For these very reasons among others it was decided to conduct a 
smaller scale pilot study using interview techniques to gather data. 
The hope of the study -xjuld be to obtain useful data on LMET effect¬ 
iveness to better design a much larger overall LMET evaluation. 

B. A PLAN OEVELOPES 

In December 1980, a recommended study design was received from 
McBer and Company based upon the pilot level study concept. 5 McBer 

5In the letter McBer and Company reiterated support for 
efforts to conduct an LMET evaluation. [McBER AND COMPANY,1981] 









recommended the strategy of a combination of open.ended interview 
questions with a survey instrument to be completed by the respondent 
immediately after each interview. Interviews were to be conducted with 
LMET graduates, two of their subordinates and an immediate super, 
visor. Suggested sample questions to be asked for each level 
[graduate, subordinate and supervisor] were included from McBer. The 
questions were designed to concentrate on behaviors [has their behavior 
changed?] and results [does it make any difference to bottom line 
measures?], rather than reactions [how do people like the course?] and 
knowledge . content [what did they learn?].6 in this way, the in 
depth interviews probing for specific, behavioral exanples from the 
LMET graduates could be validated by their subordinates and supervisor 
in an effort to obtain more meaningful data. McBer also mentioned a 
suggested coding scheme for the interview data which had been used to 
evaluate their own business leadership training program. McBer 
recommended that the graduate students be trained in interview skills 
as well as an LMET coding scheme for scoring interview responses. Their 
final recommendation was to send both researchers through the LMET 
course itself, before conducting the study. 

Through December and January the pilot study preparations 
continued. During this pre.interview phase the major task would be to 

^McBer states, "...'Reactions' are best addressed immedi. 
ately after the course [which Is indeed happening] and 'knowledge - 
content’ is currently being evaluated immediately after the course in 
response to requests from CNET." 









improve interview techniques of the two graduate students as well as to 
develop a knowledgeable background on (.MET and general Navy leadership 
training. 

In the area of interview techniques, the graduate students relied 
on a variety of sources to further develop their skills. Both graduate 
students were surface warfare, line lieutentant commander with 
similar experience in management practices developed in at-sea opera¬ 
tional environments for a combined total of nearly 20 years in the 
Navy. Additionally, both were pursuing Master of Science degrees in 
Management, specifically Human Resource Management [HRM] with an 
emphasis in Organizational Development [O.D.]?. Their experience as 
Naval officers as well as their knowledge of applied and theoretical 
concepts of management would serve as a substantial foundation in 
developing interviewing skills. 

Prior to interviewing both graduate students attempted a 2 day 
workshop conducted by Captain Phillip Butler, USN, on interviewing.8 
The workshop concentrated on developing and planning an overall 
interview strategy. Stressing the biases of this form of social 


?The Human Resource Management curriculun at the Naval Post¬ 
graduate School consists of core courses in Economics, Operations 
Research, Systems Analysis, Management, Accounting, Computers and 
Statistics followed by sub speciality courses in Organizational Theory, 
Educational Design, Labor Relations and Personnel Management, 
Organizational Development, Individual and Group Processes as well as 
Public Administration. 

^Captain Butler holds a Doctorate in Sociology from the 
University of California, San Diego. He is well known for his active 
use of interviewing as a viable means of gathering data in both 
military and civilian organizations. He was a co.advisor for the pilot 
study. 
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inquiry, he introduced specific interview methods and skills available 
as well as thoughts on their effective use. At the conclusion of the 
workshop participants were given the opportunity to team interview on a 
variety of subjects with subsequent feedback on their effectiveness. 
Using the concepts developed from this workshop, actual interviews were 
conducted for practical experience. 9 The next area of concern was 
now turned towards the LMET concept and the actual course. Research on 
the development of Navy leadership training. Dr. McClelland's concepts 
of achievement and competency notions, McBer’s LMET study, and course 
content of LMET [all previously mentioned], were undertaken to formu¬ 
late a satisfactory base of knowledge. While there was no time to 
actually attend the LMET course due to other graduate course require¬ 
ments, a trip was made to the LMET School in Coronado, California. 
During that visit both graduate students were allowed to sit in on the 
various levels of classes as well as talk to the curriculum director 
and a nunber of LMET instructors. Literature on course content, sample 
course schedules as well as general LMET information was obtained and 
later reviewed. In mid January, the research background on LMET as 
well as the interview training was completed, a meaningful plan of 
attack could now be developed complete with a schedule of milestones. 

9 LCDR D. Vandover, one of the two pilot study researchers, 
using interview concepts from CAPT Butler's workshop as well as 
additional research, designed and conducted a 1 day Interview workshop 
which was given to a class of graduate students at the Naval 
Postgraduate School. Later both students received practical training 
In conducting interviews while Involved In consulting with a local 
business in Monterey, California. These Interviews were conducted with 
eighteen owners/employees with wide educational backgrounds, and 
Included grouping, categorizing and analyzing a large amount of data. 
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C. THE PLAN 

A plan to conduct the pilot study was proposed to and approved by 
Or. Reuben Harris. Interviews were to commence later in January and 
continue through the end of March, with the hope of obtaining at least 
20 sets of interviews. April was to be set aside to analyze the data 
while May was to be used to write the final report. The major sections 
of McBers recommended pilot study design, including the questions, 
would be utilized with three exceptions: first, the survey after the 
interview would not be used. Next, McBer's coding scheme for the 
interview data would not specifically be employed, thereby eliminating 
this additional training requirement; and finally, the graduate 
students would not attend the ten day LMET course. ^ 

Dr. Harris obtained a point of contact to assist the graduate 
students in scheduling the commands to be interviewed using the Human 
Resources Management Center [HRMC] in San Diego, California, and the 
Human Resources Management Detachment [HRMD] in Alameda, California, 
for the commands in those areas respectively. It was left up to the 
graduate students to contact the center and detachment to work out the 
final details. Additionally, the cognizant officer on the staff of the 


‘®A list of the actual questions used during the study is 
contained in Appendix A. 

^All three design recommendations appeared to be good 
suggestions, however, the report was not received until mid January. 
This left insufficient time to fully develop a survey, arrange for 
additional training in McBer's coding scheme and attend two weeks of 
LMET school. In addition, due to the small sample size, it was felt 
that the survey would provide little additional insight to this 
particular study. All three recommendations deserve thoughtful 
consideration as part of a large scale evaluation. 


10 
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Commander, Pacific Fleet, [CINCPACFLT], was notified of details 
concerning the study. The types of commands and levels of LMET gradu¬ 
ates to be interviewed were not specified and were left to the judge- 
ent of the graduate students. 

It was decided by the graduate students to try to interview a 
wide-ranging sample of operational [as opposed to shore and support] 
commands in terms of warfare specialties; that is air, submarine and 
surface, with emphasis on the latter due to their familiarity and 
experience. Since the emphasis would be on surface ships the sample 
would be further divided into carriers, cruisers and destroyers, 
amphibious, auxiliary support and miscellaneous. Each of the three 
major Pacific Fleet geographical areas would be sampled, San Diego, 
California; Alameda, California; and Pearl Harbor, Hawaii. Among the 
selected commands, it was decided to interview LMET graduates in three 
specific billet levels, each corresponding to LMET levels of petty 
officer [E-6], chief petty officer [E-7 thru E-9], and division 
officer [0-1 thru 0-4]. Department head graduates as well as executive 
and commanding officers would not be interviewed, except as immediate 
supervisors to the LMET graduates. On January 29, 1981, the first in¬ 
terviews for the pilot study were conducted onboard a surface ship in 
Alameda, California. Nearly two months later the last interview would 
be completed onboard another surface ship in Pearl Harbor, Hawaii. 

Actual selection of the commands to be Interviewed was left to the 
discretion of the HRMC and HRMD. The only guidance given was that the 
wider the variety of operational commands the better. Undoubted a major 
consideration was which commands were Inport. Any other criteria that 
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the HRMC/D considered in selecting commands is not known, however, 
there remains a possibility that other selection criteria may have been 
used and may affect the data. 

One to two weeks prior to the interviews the point of contact in 
the desired area was called and asked to nominate or identify a command 
for each of the two days the interviewers were planning to be in the 
area. Once the commands had been identified, [points of contact at the 
command and telephone numbers were supplied by the HRMC and HRMD], the 
interviewers would contact the command, usually through the executive 
officer, to explain the proposed visit and confirm a date and the 
commands actual location. This selection procedure continued smoothly 
throughout the interview phase with relatively few problems. It was 
easy to tell if the HRMC or HRMD had actually contacted the units or 
just identified them based upon an employment schedule, commands 
personally contacted by the HRMC and HRMD were far more enthusiastic 
about the proposed visit than were those not contacted. While a 
considerable amount of extra time was necessary to "convince and sell 
the visit" to unaware commands, no one ever refused to visit. 

The initial telephone contact with the commands by the 
interviewers generally lasted 15-30 minutes. This was a sufficient 
length of time to explain the specific details of the visit. The visit 
was explained as a study on LMET effectiveness for the specific purpose 
of completing a Masters Thesis at the Naval Postgraduate School. Upon 
realizing that the interviewers were Navy surface warfare officers. 
Instant credibility seemed to be established, resulting In an informal, 
open door policy for the visit. One executive officer remarked, "Hey, 
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anything you guys need-it's good to see the Navy using its own like 

this." The executive officers were informed of the desire to interview 
3-5 LMET graduates [within the three levels previously discussed], 
along with their immediate supervisor and two subordinates. They 

were told that the interviews for the LMET graduates would last 
approximately one hour, while subordinate and immediate supervisor 
interviews would only take a total of 30 minutes. Initially, the 

executive officers were allowed to set up their own schedule for the 
days interviewing. However, as the study progressed a schedule 
developed by one of the commands became the recommended example that 
executive officers were encouraged to follow.^ The only 

requirements stipulated by the interviewers was the need for two 
separate spaces or rooms [relatively quiet if possible], which would be 
used to actually conduct the interviews. The executive officer was 
then informed of the interviewers desires to wear civilian clothes as 
part of the study, when applicable, no one refused: 

Actual selection of the LMET graduates at each command was left to 
the discretion of the executive officer. It is not known how the 
selection process actually occurred in each instance, however, it is 
known that some graduates volunteered while others "were volunteered" 

^After interviewing at several commands the two subordinate 
requirement was changed to include only one immediate subordinate. 
This resulted in a time savings. 

^McBer had suggested 1.5 hours for LMET graduates and 45 minutes 
for subordinates and supervisors. As the study progressed, actual 
interview times for LMET graduates rarely lasted 60 minutes, with sub 
ordinate and supervisor interviews easily completed within 15 minutes. 

^The sample schedule used is contained in Table 2. 
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TABLE 2 

SAMPLE INTERVIEW SCHEDULE 


The following personnel are scheduled for interviews on (date) at 
the indicated times. Interviews will be held in the following two 
locations: 



to participate. The executive officer was not requested to specifical¬ 
ly nominate certain levels of individuals since the totals among the 
three levels remained fairly equal throughout the entire study. 

Several days prior to the actual visit the commands were again 
contacted to remind them of the time and date of the visit as well as 
to recheck the actual location of the ship. While problems were few 
and far between, they were all resolved during this final contact. On 
one occasion, a forced cancellation by a command was required due to an 
upcoming inspection. Using personal contacts by the interviewers, 
another command was substituted causing no problems. 

All interview visits except one were scheduled during the normal 
work week, with the vast majority occurring on Thursdays and Fridays. 
While the weekends were certainly more convenient to the interviewers 
due to graduate course studies, they were incredibly inconvenient for 
the commands for obvious reasons. The one exception occurred through 
special arrangement over a weekend after it was learned that a 
satisfactory sample [five graduates, supervisors, and subordinates], 
could be obtained within the duty sections over a two day period. 

Arrival at the commands on the day of the interviewing was usually 
between 0745-0815. Although the individual commands generally start 
their work day earlier, it was felt that it would be more convenient 
and less hectic to arrive after officer's call, giving the executive 
officer as well as the ship a chance to start their day before our 
intrusion - the commands agreed unanimously. On every occasion the 
quarterdeck watch had expected the arrival and the Interviewers were 
taken to the executive officer. After introductions and the 






interviewing plan had been discussed the executive officer would 
typically ask, "..So what have you found out about LMET that we should 
know?" On several occasions the interviewers were asked to meet the 
commanding officer where he also expressed an interest in learning what 
had been found to date.^® 

D. THE INTERVIEW PROCESS 

Interviews were first conducted with the LMET graduates using 
both interviewers. One interviewer would question the respondent while 
the other interviewer would passively take notes as well as handle the 
tape recorder. The respondent was introduced to the interviewers and 
made to feel relaxed. He was told specifically about the study and 
assured of confidentiality concerning his responses/'® He was told 
that if he was to be quoted there would be absolutely no mention of 
either his name or his command. His permission was solicited to allow 
the use of the tape recorder during the interview. Without exception, 
permission was always received, allowing for a complete taping of all 
interviews.He was further asked not to mention any names of 
individuals during the interview. 

15 

Commanding officers and executive officers of the commands 
interviewed were generally interested in any information concerning 
LMET. The interviewers were careful to avoid any evaluative remarks 
under the guise that it would not be known until all interviews had 
been completed with the data analyzed. 

^®A11 51 interviewed LMET graduate' were male except in one 
exception a female division officer. The use of the masculine pronoun 
throughout the research denotes the entire sample including this 
exception. 

'^All tapes contain an interview nunber as well as the commands 
name. Respondents are identified only as to their LMET level [petty 
officer, chief petty officer or officer] as well as their relationship 
[graduate, subordinate or supervisor]. 
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The graduates were all asked the 17 questions [Appendix A] origin- 

18 

ally recommended by McBer. Additionally, they were asked the 

following four questions: 

1. How long ago did you graduate from LMET school? 

2. Did you attend LMET school prior to arriving at this 
command? 

3. Would you send anyone to LMET school? Why? 

4. Is there anything else you would like to tell us con- 
concerning LMET? 

The interviews with the graduates proceeded smoothly. The only 
minor problem was a confusion between LMT and LMET. Seven respondents 
identified by their commands as LMET graduates had actually attended 
LMT school. These interviews were not included in the study results 
reported and were terminated as soon as this fact was uncovered. 

Interviews with the graduates immediate subordinate and supervisor 
were conducted after the graduate's interview, usually later that 
afternoon. In this way they could be used to validate the graduates 
remarks. Interviews were conducted by a single interviewer, with two 
interviews ongoing simultaneously in separate rooms. Identical intro¬ 
ductions were made as had been done with the LMET graduates. 
Confidentiality was promised and respondents were asked to omit names 
using only "him" or "he" to identify the graduate. Permission was 
obtained in every case to record the interviews on tape. 

]8 In some instances several LMET stimulus questions were 
specifically omitted if the interviewers believed they had already been 
answered in discussing previous questions. 
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The subordinates as well as the supervisors were all asked the 
four questions recommended by McBer. Additionally, they were asked the 
following four questions [note the difference in questions 2 and 3 
depending upon the respondent]: 

[Both] 1. What do you like or admire most about his [the 
graduates] performance onboard? 

[Subordinate] 2.a Would you work for him again? Why? 

[Supervisor] 2.b Would you work for him [if roles were reversed]? 

Why? 

[Subordinate] 3.a Would you like to go to LMET school? Why?^ 9 

[Supervisor] 3.b Would you send anyone to LMET school? Why? 

[Both] 4. Is there anything else you would like to say 
concerning this individual or LMET school? 

In some of the earlier interviews with these individuals it was 
noted that they did not proceed as smoothly as had the graduates inter¬ 
views. There was a tendency on the subordinates part to remain intro¬ 
verted and to assume that the interviewers were trying to personally 
grade or evaluate the graduate. Their perception seemed to be that 
their information could have detrimental con sequences on either them¬ 
selves or the LMET graduate. Once this was noticed, early in the study, 
an effort was made prior to the interview to explain to them that the 
information they would give would be used only to categorize the LMET 
graduate among other LMET graduates. Interviews with supervisors 
proceeded smoothly with no problems. 

19 If it was found that the subordinate had gone to LMET school, 
then question 3.b. was substituted. 
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At the conclusion of the day, after all interview had been 

completed, the interviewers then met to discuss the days events. 
Preparations for the next interviews were begun using any new lessons 
learned from that days activities. 

The interview process as discussed above was repeated through 
March in a fairly methodical fashion. In the end, the pilot study 
sample size consisted of 51 LMET graduate interviews evenly divided 
among the three junior levels of LMET training [LPQ, CPO and DO]. The 
sample represented 13 operational commands within the 3 warfare areas 
in three Pacific Fleet geographical locations.20 

During the next four weeks the tedious process of analyzing the 
data was undertaken. Responses to each question were carefully 

analyzed using the notes and tapes taken during the interviews. All 
data was team analyzed using both students rather than each separately 
analyzing the data twice. A coding scheme was developed for each 

question prior to analyzing the data, in an attempt to categorize the 

data into a somewhat more usable format. Although extremely time 
consuming, the interviews analyzed using the LMET [McBER] competencies 
produced an enormous amount of information concerning both the LMET 
program in general as well as specifics on LMET graduates. 


20summary results of the sample are contained in Table 3. 
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TABLE 3 

PILOT STUDY SAMPLE 


I. BREAKDOWN OF SAMPLE BY COMMAND LOCATION 


Type of 


Graduate 

Interviewer's 

Command 

Location 

Rate/Rank 

Appearance 

Service Force Ship [AFS] 

A1 ameda 

SK1/ENS/BM1/RM1 

Civil ian* 

Amphibious Ship [LPD] 

Alameda 

LTJG/ET1/ENS/SKC 

Civilian 

Nuclear Submarine [SSN] 

San Diego 

RMC/ST1/IC1/MMC/ETCS 

Civilian 

Small Craft Unit 

San Diego 

BMC/SK1/BM1/HM1/ENCS 

Civilian 

Service Force Ship [AOR] 

A1 ameda 

MSC/RMC/EMC/LTOG/HTC 

Uniform 

Service Force Ship [AFS] 

A1 ameda 

ENS/DP1/QMCS/RMC 

Uniform 

Submarine Rescue Ship [ASR] 

San Diego 

LT/ENCS/EN1 

Uniform 

Aviation Squadron [RVAW] 

San Diego 

LT/LCDR/LT/ENS 

Uniform 

Aircraft Carrier [CV] 

A1 ameda 

MM1/ENS/AG1 

Uniform 

Tender [AR] 

San Diego 

BM1/HTC/ET1/BM1/HTC 

Uniform 

Destroyer [OD] 

San Oiego 

DSl/GMGT/OSC/GSEC/ENS Civilian 

Fast Frigate [FF] 

Cruiser [CG] 

Pearl Harbor ENS/ENS 

Uniform 

Pearl Harbor ENS/LT 

Uniform 

*23 interviews in civilian 

II. BREAKDOWN OF SAMPLE BY 

clothes, 28 

PAYGRADE 

in uniforms 


Paygrade of LMET Graduate 

E6 E7 

E8 E9 01 02 

03 04 

(Rank) 

P01 CPO 

SCPO MCPO ENS LTJG 

LT LCDR 

TOTALS 

18 13 

4 0 9 2 

4 1 

LMET Level of Graduate 

E6 

E7-E9 

01,04 

Number in Sample (X) 

18 [35.3] 

17 [33.3] 

16 [31.4] 


TABLE 3 [con't] 


III. BREAKDOWN BY WARFARE SPECIALTY/RATE 


11 Surface Warfare Officers 
1 Supply Corp Officer 
3 Aviation Officers 
1 Aviation Maintenance Officer 


Supply Dept. 

Medical 

Weapons/Deck 

Aviation 

Engineering 

Operations 

3 SK’s 

1 HM 

5 BM's 

1 AG 

1 IC 

4 RM's 

1 MS 


1 ST 


2 MM's 

3 ET's 

1 DP 


1 GM 


3 EN's 

1 qM 





1 EM 

1 DS 





3 HT's 

1 OS 

— 

- 

■«— 

- 

_1_ GSE 

. 

5 

1 

7 

1 

11 

10 


IV. BREAKDOWN BY TIME SINCE GRADUATION FROM LMET SCHOOL 


Officer CPO POI 


Less then 6 mo. 4 

6-12 mos. 5 

1_2 years 7 

Over 2 years 0 


6 9 19 [37.3*3 

7 3 15 [29.4X] 

4 4 15 [29.4*3 

0 2 2 [ 3.9X] 

ST [100.OX] 
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V. ANALYSIS AND FINDINGS 


The data gathered from the pilot study can be grouped into two 
broad areas. First is the data indirectly gathered through observing 
and participating in the interview process , that is, information on and 
resulting from the methodology used. Second is the actual content of 
the interviews, that is, data concerning the LMET graduate to the LMET 
Program. These data findings can be used to judge the value of the 
pilot study technique as a preliminary evaluation plan for major 
program, specifically LMET in this case, prior to committing extensive 
resources towards a full-scale evaluation. 

A. INTERVIEW ANALYSIS 

Before discussing actual interview content and LMET findings, let 
us look first at the process used to specifically gather this 
information. There exists many advantages as well as disadvantages 
[discussed earlier], in using interviews to gather data. The quality 
of data [both in depth and range] obtained from interviews is normally 
offset by the amount of time required in using this method. Once the 
interview method has been examined in relation to this study, the LMET 
content findings will be presented. 

1. Interviewing: A Time Consuming Process 

The interviewing process was extremely time consuming, sub. 
stantiating previous research claims to this fact. Each LMET graduate 
was team Interviewed for approximately 45 minutes by an active and 
passive observer for a total expenditure of 2.25 man.hours. Each, 







immediate subordinate and supervisor to the LMET graduate was inter, 
viewed for approximately 15 minutes, one.on.one for a total expenditure 
of 1.0 man.hour. A simple arithmetic computation reveals that 3.25 
man.hours were expended per LMET graduate set of interviews. 
Theoretically, this times the 51 interview sets equals a total of 
165.75 man.hours. In actuality, due to interviews lasting longer than 
45 minutes, approximately 180 man.hours were expended in data 
collection. This only accounts for actual time expended in 

interviewing and does not include any travel, or set-up time for the 
interviewers nor time spent arranging for the interviews. It is worthy 
to mention that considerable time and travel money was conserved 
through interviewing several graduates per command [3 to 6 graduates 
with 5 the ideal nimber for 1 day].^ 

This lengthy expenditure of time in having the respondents 
full attention was a primary advantage in obtaining a wide range and 
depth of information. It allowed ample opportunity to exploit the 
respondents' desire to voice their own feelings concerning the LMET 
training, as well as specifically, how LMET was seen to be emphasized 
at that command. Those respondents that were extremists towards the 
training were quick to point out its weaknesses or strengths while 
those that were neutral presented a more toned down version. The end 
result was a good overall picture of how LMET graduates behaved and 
felt about their training within the sample. 


sample interview schedule is contained in Table 2. 
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2. In Depth Process Analysis: The LMET Interview 

This section contains general remarks on the LMET interview 
process followed by examining five specific areas concerning: interview 
flexibility, the interview schedule, the interview environment, the 
issue of military vs civilian interviewers, and finally comments 
concerning interview introduction. The data consists of 69.25 hours of 
taped conversation in a leading question, free answer format. As such, 
the information is considered to be soft-data, in that it is difficult 
to quantify. Often it was seen that two people can use very similar 
terms to describe the same subject yet mean very different ideas. Non- 
verbalized expressions such as tone of voice, hand and body movements, 
and facial expressions served to emphasize points and at times 
contradict what was actually being stated. This caused the interviewer 
to explore deeper while taking written notes concerning the 
respondent's attitude and behavior. This resulted in data consisting 
of not only the taped conversations, but also the notes and feelings of 
the interviewers. Even after 51 interviews it was still possible to 
vividly remember each respondent and their general attitude while 
listening to the tapes as much as four months after the interview. In 
addition to the interview questions, the respondent was given the 
opportunity to make any additional consents that might be pertinent or 
of felt importance. This resulted in a sizable amount of not easily 
catagoried responses that added to the total information obtained. 

The subjectiveness of this soft data was further accented by 
the Interviewer's own biases towards the LMET Program. This bias tend¬ 
ed to cause an interviewer to probe deeper into certain areas where the 
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respondent was reinforcing the biases of the interviewer and perhaps 
cutting off or playing down other those in opposition. As the 
interviewing progressed, the interviewers started to observe what were 
thought to be trends and may have tended to guide respondents into 
confirming those ideas or trends. Oppossing these tendencies were the 
interviewer's knowledge of their own biases and a professional interest 
in suppressing them as much as possible, 
a. Interview Flexibility 

One of the early advantages that became apparent in using 
the interview, as opposed to a survey, was its flexibility in gathering 
data. The interviewer was able to control the direction of the infor. 
mation flow through questions and requests for clarification of 
statements. It quickly became obvious that the interview questions had 
a natural progression that made asking some unnecessary as the answer 
would be given in discussing a previous question. This resulted in 
rearranging the questions in this natural order and merely encourag¬ 
ing the respondent to continue until all of the desired data had been 
obtained. In not breaking the respondent's train of thought, the 
interviewing process progressed smoothly and quickly, allowing ample 
opportunity for the respondents to voice their thoughts on any given 
topic or question. While this did allow for the respondent to answer 
each question fully, it also led to some detailed examples and 
explanations. 
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The opportunity of interaction between interviewer and 
respondent ensured that the interviewer was correctly interpreting re. 
sponses. In rephasing further questions, and asking for clarification 
of specific responses, the interviewer had ample opportunity to clear 
up any misconceptions or hazy areas identified by the respondent during 
the interview. Similarly a distinct advantage of the interview was the 
ability to rephase questions to ensure the respondent understood what 
was being asked. This exchange of ideas led to the rewriting of 
several questions that were ambiguous or unclear as to what they were 
asking. It was discovered that what seemed to be a perfectly clear 
question to either interviewer was often not understood by a respondent 
of a different educational or cultural background. 

As a final note on this subject, the very flexibility of 
the interview process required the undivided attention of the 
interviewer. This results in a tendency of the interviewer to "burn 
out" after several consecutive days of interviewing. The interviews 
were conducted weekly over a two day period, normally a Thursday and 
Friday. If a full five days of interviewing were conducted there may 
have been a difficulty for interviewers to maintain their interest and 
enthusiasm for the project. The consequence is obviously a risk of 
gathering lower quality data at the end of a week as compared to the 
beginning. 

b. Interview Schedule 

An interviewing schedule was developed and used soon after 
the data gathering phase had begun. For the first two commands no 
schedule was provided, only a description of what was necessary, 










namely to interview four to six LMET graduates as well as an immediate 
subordinate and supervisor for each. The first command devised and 
promulgated a schedule [as well as a Plan of the Day [POD] note], that 
worked so well [see Table 2] that it was recommended for use to all 
subsequent commands. The use of this standardized schedule allowed the 
interviewers to develop a routine that minimized wasted time, and 
increased efficiency in conducting the interviews. Additionally, most 
command XOs appeared grateful to receive a schedule format in which 
they could merely insert names. It reduced the administrative burden 
to the commands as well as completely satisfying the interviewer's 
needs. The sample schedule also minimized disruptions for their crew 
by promulgating beforehand, an interview schedule complete with inter¬ 
view location and time. 

The schedule that was developed by the first command had 
all of the LMET graduates interviewed first, in the morning, followed 
by their supervisors and subordinates in the afternoon. The outcome 
could not have been better planned. Through interviewing all of the 
LMET graduates first, the interviewers formed a crude idea of the 
command's attitude towards the LMET Program. It also allowed the 
interviewers to categorize each LMET graduate as to their degree of 
adoption to the LMET competencies. Seldom did the LMET graduate 
interviews exceed the allotted 60 minutes which allowed the inter¬ 
viewers [active and passive] to quickly exchange impressions on each 
interview while still fresh in their minds. 
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The most beneficial aspect of interviewing the LMET 
graduate first was the detailed data on behavior gained, which could 
later be validated by their supervisor and subordinate. In a few 
cases, when the supervisor or subordinate was interviewed first, only 
general responses could be solicited in the hope that they would in 
fact substantiate what the graduate would say. The interviewing 
process operated best as a validation of behavior when conducted in the 
order of LMET graduate, then either subordinate or supervisor, 
c. Interview Environment 

All interviews for this pilot study were conducted either 
onboard the unit or working spaces and areas under the direct control 
of the command. This kept the respondent on familiar ground, in what 
could be termed as "home territory". However, whenever possible a 
neutral zone or area was used in order to remove any feelings of "big 
brother watching you". For enlisted personnel this seemed to mean any 
space where officers did not have ready access and included such areas 
as unoccupied officer staterooms, unit commander's offices [unit 
commander not embarked], and recreation rooms or librarys. In one 
instance the wardroom was used with obvious detremental effects. 
Enlisted respondents appeared to be uneasy and nervous, and stopped 
talking to look and see who entered every time the door opened. For 
officer personnel, any place appeared to be sufficient. In remaining 
onboard the command while conducting the interviews it appeared to be 
less disruptive and more ’Ikely for personnel to respond promptly to 
the promulgated time schedule. In fact, throughout the study, inter¬ 
viewers seldom had to wait more than a few minutes and respondents 










seldom had any wait at all. The smoothness with which the schedule 
operated was in no small part responsible for the large number of 
interviews that were able to be conducted. 

All interviews were tape recorded instead of depending 
entirely on written notes. The reason for the use of the tape recorder 
and the confidentiality of the interview were carefully explained to 
the respondent prior to each interview. After obtaining the respond¬ 
ent's permission to tape the interview an identification statement was 
made on the tape. The statement specifically did not include the 
respondent's name or rate to reverify the confidentiality that had been 
promised. With this formality over, the interview proceeded with 
little or no attention placed on the tape recorder. It did not appear 
to inhibit any person from expressing their opinions. The tape re¬ 
corder could be slightly disruptive however, if a tape ended in the 
middle of an interview. The respondent would normally stop talking 
until a new tape was started even when the interviewer insisted that he 
continue and not worry about the tape. 

Maintaining the neutrality of the interviewing environment 
is extremely important as was demonstrated during the course of two 
interviews in which the supervisor was nearby. In one instance, the 
supervisor was at his desk on the other side of an equipment rack, and 
in the other, he was actually being interviewed simultaneously across 
the room. In both instances, the respondent's reactions were very 
similar. The respondents appeared to be nervous and continuously 
glanced In the direction of their supervisor. One respondent's replies 
to questions seemed to be for the benefit of his supervisor. The 
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respondents seemed to realize that the confidentiality of what was 
being said could not be guaranteed under these circumstances, 
d. Interviewer - Military vs Civilian 

Concerning the issue of military versus civilian inter¬ 
viewers, a couple of points are of note. In 23 interview sets civilian 
clothes instead of military uniforms were worn to see if that had any 
effect upon responses. It appeared to the interviews that the wearing 
of uniforms was optimal over civilian clothes. Instant establishment 
of credibility as a person knowledgeable in Navy matters exsisted when 
wearing a uniform that seemed to increase the information flow. When 
civilian clothes were worn there was a noticed tendency for the 
respondent to explain routine procedures. Navy acronyms, and common 
programs, that were not done when interviewing in uniform. In both 
cases, using familiar vocabulary and Navy jargon greatly assisted in 
conducting the interviewing process. Additionally, a feeling of 
empathy for each others positon was quickly established when in 
uniform. The rank and age of the interviewers also seemed to 
facilitate the entire process. Lieutenant or lieutenant commander is 
sufficiently senior to develop good rapport with senior petty officers 
and chief petty officers, while a fairly youthful appearance [age 
30-31, one interviewer with a beard] reduced the "generation gap- that 
might have inhibited discussion with younger officers and enlisted 
personnel. The fact that enlisted personnel are quite used to 
interfacing with officers but not with civilians in an official 
capacity probably had a positive impact on interviewing in uniform. 
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e. Need for Interview Introduction 

A final issue that surfaced during the interviewing 
process was that of the purpose. Some subordinates seemed reluctant to 
talk about their supervisor despite assurances of confidentiality. At 
one point an interview was briefly stopped and it was explained care¬ 
fully to the respondent that the intent of the questions and study was 
not to evaluate his supervisor's performance in itself, but rather to 
evaluate a program of leadership/management techniques. With these 
assurances the interview resummed in a much more relaxed and co-opera¬ 
tive atmosphere. After this incident, subordinate interviews were 
begun by including this performance concept as part of the overall 
interview introduction with good results. 

3. Interview Remarks 

The interview process provided an excellent method for 
gathering data from the respondents. Primarily through its flexibil¬ 
ity, the researchers were afforded the opportunity to explore relevant 
areas of interest as well as uncover tangential issues of concern. 
Concluding interview remarks concerning a future full-scale evaluation 
are contained in the next chapter. The remainder of this chapter 
concerns ’•.he specific LMET content information derived from the 
interviews themselves. 

8. FINDINGS CONCERNING LMET 

The findings of this study concerning LMET must be viewed with 
full recognition of the limitations of the sample. The sample in¬ 
cluded only 51 LMET graduates selected from 13 operational commands in 
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three west coast geographical areas. Furthermore, the sample was not 
random and therefore cannot be said or alluded to be truly representa¬ 
tive of the Navy. However, the sample did contain a wide variety of 
warfare specialties, ranks, rates and operational commands, and does 
provide a good insight to LMET's influence, particularly on the sample 
population. 

The method of describing the LMET findings which will be addressed 
in this analysis is using the Kilpatrick paradigm consisting of four 
levels or domains of learning. These levels are: 

1. Reactions [How do people like the course?] 

2. Knowledge-content [What did they learn?] 

3. Behavior [Has their behavior changed?] 

4. Results [Does it make any difference to bottom line 
measures?] [McBER.1981] 

The level of "reaction" is the subjective opinion of the LMET graduate 
towards the overall course usefulness. This data can be gathered at 
any time, but is probably best addressed immediately after the course 
[this is done via a feedback survey completed upon completion of LMET 
training]. "Knowledge-content" concerns the level of retaining 
information, ideas and concepts. It also is best measured imnediately 
upon completion of the LMET training [which is occurring in response to 
CNET requests]. "Behavior" is the level in which the instruction is 
actually practiced on the job by the LMET graduate. This is the level 
in which the pilot study interview method was focused upon in 
attempting to measure LMET effectiveness. The final area of "results" 
concerns that level where the changed behavior of the LMET graduate 
does in fact bring about the changes within the system which are de¬ 
sired. This is in the realm of a cause and effect relationship and 
amounts to a prime indicator of the effectiveness of the LMET Program. 
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1. Reactions 


The first of the four levels to be examined is that of 
reactions. While it was not the intent of the study to explore this 
level, responses were obtained that provide some insight into the LMET 
graduate's "reactions" to their training, 
a. LMET Viewed as Necessary 

When querried as to whether an LMET graduate would send 
his personnel to the school, *he responses were positive [only one 
negative officer] with three general response areas given most often. 
First, was the desire to increase knowledge in the area of leadership 
theory and to increase learning and vocabulary in the field of 
management. Second, was the idea of personal improvement through a 
further awareness, either to "get-well" or to make good leaders better. 
Third, is that individuals are required to go through LMET school via 
PCS orders, that is, it is an expected training evolution for career 
personnel. To discuss these three areas in more detail, the data [see 
Table 4] indicates that an overwhelming majority of sampled graduates 
view LMET school as a useful training evolution. Specifically, the 
training was reported to provide a general background of leadership/ 
management theory complete with a specific vocabulary. This new 
vocabulary allows the graduate to be able to converse easier with 
supervisors and subordinates and often times provides names and labels 
to leadership styles and managerial techniques which they have already 
been practicing. The next perception held by graduates, appears to be 
that LMET training can provide poor leaders with the tools necessary to 
become good leaders while good leaders can become even better. Behind 
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TABLE 4 

REASONS FOR ATTENDING LMET SCHOOL 


Main Interview Reasons Given LMET Graduate Supervisor Subordinate 

OFF/CPO/POI/Totals (Totals) (Totals) 


Acquire management techniques 5/ 

Learn different leadership 
styles 2/ 

Increase "self-awareness" 4/ 

Better leadership skills 
or to "get well" 1/ 

Learn how to "get along" 
with people 0/ 

Learn how to influence people 
in a positive manner 1/ 

Build self-confidence 0/ 

Other* 2/ 


6/ 

5/ 

J6 

U 

J5 

3/ 

1/ 

_6 

_5 

_7 

0/ 

3/ 

_7 

_5 

]2_ 

2/ 

1/ 

_4 

_2 

A 

0/ 

1/ 


_2 

J_ 

0/ 

0/ 


A 

J_ 

0/ 

1/ 

_1 

A 

A 

6/ 

6/ 

14 

mm 

rnm 


Additional Respondent Comments: 


< 

i 



. 3 LMET Graduates, 6 Supervisors and 2 Subordinates stated that some 
form of LMET training should concentrate on E-4 and above instead of 
E.6 and above. 

- Officer LMET Graduates had the widest range of remarks as seen below: 
"....it [LMET School] was of little practical value." 

"-a ticket punch." 

“_a waste of time." 

"-can't change behavior." 

"....need some shipboard experience first." 

"-it was a good experience." 

"-prepared me for a leadership role." 

"....I now understand people better." 

"-1 learned how to organize my time." 

"-it was good background enrichment." 

"....it points out strengths and weaknesses." 


♦Comments in this category varied and were generally favorable toward 
LMET School. Total does not include 1 Commissioned Officer who voiced a 
negative opinion. Supervisor/Subordinate columns will not sum to 51 
due to the wide variety of responses received and not included in the 
table. 
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this notion may be a feeling of inadequacy or lack of time in which 
fleet managers and leaders feel unable to provide on-the-job leader¬ 
ship/management training. The final perception was that LMET training 
is a promotion hurdle that has to be passed, that is, the training is 
career enhancing. 

b. Why LMET? 

In addition to questioning the graduates, supervisors and 
subordinates were similarly questioned as to what their thoughts were 
concerning their desire to attend LMET. Most of the responses from 
individuals that had not attended the LMET training concerned 

perceptions about what LMET could do for them. The first of three 

general perceptions was a feeling of inadequacy in non-LMET personnel 
[especially junior enlisted] to assume positions of leadership and 
management without formal training. Second, was the feeling of needing 
additional ideas on how to lead and manage others, that is, their 
actual skills needed expanding. The third impression was that LMET 
training would provide the "right way" to lead and manage others. 

While they were not able to relate these ideas to the LMET school 
curriculum itself, it does imply the type of reputation that the 

program possesses. Fleet personnel look upon the LMET training in a 
positive light. They expect LMET to improve their leadership/manage¬ 
ment skills as well as to be provided with the "right way" to lead and 
manage others. In summary, the reaction to LMET training seems to 
receive favorable considerations from both graduates and non-graduates 
alike. As stated by one respondent, "It [LMET training] certainly 
can't hurt!" 
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2. Knowledge-Content 


The second level or domain to be addressed is that concerning 
what the LMET graduates actually learned. They are tested on this 
level prior to leaving the school and the interview process did not 
specifically seek data on it, nevertheless several insights were 
revealed. 

a. LMET Knowledge 

For those graduates who had been away from the school for 
a length of time [6 months or longer], there were indications that they 
had forgotten some of the specific content-knowledge of their LMET 
training. However, as the interview progressed, they tended to recall 
some of these specifics and began to respond to interview questions 
using the LMET school vocabulary. The actual interviews appeared to 
surface training concepts which initially appeared to have been 
forgotten. 

b. Setting Goals and Performance Standards 

Of the five competency clusters taught during LMET school, 
the data indicates that the competency group of efficiency and 
effectiveness seems to be the least understood and practiced. Concern¬ 
ing efficiency and effectiveness, it can be paraphrased as, "Doing 
things well, and wanting to do better". [HRMC,N.D.] It includes 
setting goals and performance standards. 

8ehaviors demonstrated by one who sets goals 

and performance standards include the following: 
♦Establishes specific work goals 
♦Is concerned for standards of task performance 
♦Revises goals to make them realistic 
♦Sets deadlines for task accomplishment [HRMC.N.D.] 











As indicated in Table 5 and 6, LMET graduates expressed adequate 
knowledge concerning goal setting, establishing performance standards 
for subordinates and clarifing evaluation standards. However, when 
specifically asked whether they infact practiced these behaviors the 
response was negative as also validated by their subordinates. The 
implication is that this competency cluster remains at the 
knowledge-content domain. It would appear from the sample that LMET 
graduates do not routinely practice setting goals and performance 
standards for their subordinates. Additionally, LMET graduates sampled 
do not seek clarification from their supervisors as to goals and 
expectations that apply to themselves. As best expressed by an LMET 

petty officer graduate "-1 guess I don't really tell them what I 

expect...I know I should." 

In sunmary, the data analyzed from the interviews within 
the 13 commands indicates that sampled LMET graduates are knowledgeable 
concerning the content of their training. However, there appears to be 
a lack of application in certain areas. A lack of application that 
some freely admit. 

3. Behavior 

The pilot study research was directed specifically at this 
third domain - the behavioral level, that is, whether the LMET gradu¬ 
ates exhibit any behavioral changes. Overall, there were no systematic 
behavioral changes that ran across the sample. However, there were 
isolated behavioral changes which were clearly the result of LMET 
training. For those several cases where a sharp and drastic change 
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TABLE 5 

GOAL SETTING BY LMET GRADUATES 


The question asked to LMET graduates was, "What goals did you set in 
LMET school? Follow-up questioning and probing attempted to determine if 
goal setting was used routinely as a management tool. 


Response to Initial Question 

Officer 

CPO 

P01 

1. Measurable goal dealing with 
immediate future leadership/ 
management position 

4 

7 

8 

2. Personal goal 

2 

5 

5 

3. Vague goal involving some 
phase of leadership [i.e. 
"achieve good leadership", 
"improve leadership style", 
"become a better manager"] 

2 

4 

3 

4. Forgot/don't remember 

J3 

J_ 

_2 

TOTALS 

16 

17 

18 


Follow-up question results: 

Officer 

CPO 

P01 


Number of LMET graduates who have 
established work group goa1s[%] 

9/16 [56] 

16/17 [94] 

16/18 

[89] 

Number of LMET graduates setting 
measurable goals[%] 

4/16 [25] 

8/17 [47] 

9/18 

[50] 

Number of LMET graduates setting 
goals with time frames[%] 

0/16 [0] 

3/17 [18] 

6/18 

[33] 








TABLE 6 

SETTING PERFORMANCE STANDARDS BY LMET GRADUATES 

The question concerning performance measures was, "How is your per¬ 
formance on the job measured?" Additional probing was aimed at estab¬ 
lishing specific measures. A rephrasing of the question to, "How do you 
know when you're doing a good job?", was particularly effective. 

I. LMET Graduate Self-Report Response* 


Performance Measurements 

OFF 

CPO 

P01 

Task accomplishment ["get the job done"] 

13 

13 

14 

Technical proficiency in rate or specialty 

4 

6 

3 

Manage the work group [emphasis on smooth¬ 
ness of operation] 

1 

5 

3 

Do not know 

2 

2 

4 

Amount of feedback provided supervisor 

4 

4 

1 

Effective training program 

1 

2 

1 

Establishing good inter-personal re¬ 
lationship with subordinates 

2 

0 

1 

Goal setting 

2 

0 

0 

Prioritizing workload properly 

2 

3 

0 

Keep supervisor out of trouble 

1 

1 

2 

Appearance of workgroup personnel 

1 

1 

1 

Material Resource Management 

1 

1 

1 

Morale of work group personnel 

2 

1 

0 

Keeping up with administration [paper work] 

2 

0 

0 

Amount of flexibility 

1 

0 

2 

Monitoring work group 

0 

2 

1 

Work group planning 

0 

3 

1 


*Colturns may total to more than the nunber of graduates due to multiple 
responses. 
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TABLE 6 [con't] 


II. Validated LMET Graduate Performance Measurements* 



Supervisor 

Validation 

LMET 


Agree with 
graduate 

Disagree with 
graduate 

graduates 
not knowinq 

Officer 

9 

3 

2 

CPO 

10 

3 

2 

P01 

9 

4 

4 

Totals: 

28 

10 

8 


III. PERCEIVED 

PERFORMANCE STANDARDS 

Oo not know 

OF SUBORDINATES TO LMET 

Task Accomplishment 

GRAUDATES 

Others 

Officer 

4 

7 

1 

CPO 

4 

9 

4 

P01 

3 

12 

2 

Totals: 

11 

28 

7 


♦This tabulation presents the LMT graduates perceived performance 
measurements as opposed to the immediate supervisor's stated performance 
measurements. Totals may not equal number of graduates due to supervisor 
or subordinate not interviewed for this question. 









took place the personnel were clearly looking for help.^ They 
attended LMET school as a "get well" measure with a substantial felt 
need to improve. In general, those that exhibited the hoped for 
behaviors or competencies indicated that they did so prior to LMET 
school, while those not exhibiting the behaviors desired did not see 
the value of those competencies, feeling they were not applicable in 
their specific situation. 

a. Seniority among LMET Graduates 

The data seemed to indicate that there may be a 
correlation [negative] between senority and extent of behavioral 
change. As Table 7 seems to Indicate, chief petty officers are less 
affected by LMET training as far as changing their behavior. While 


there is some subjective indication that they in fact can better 
appreciate what is being taught and how it applies due to their 


experience, it seems that they tend to be more skeptical towards new 

ideas. The tendency seems to be, to use the LMET training as a 

reinforcement of their own leadership/managerial style by adhering to 

those competencies that reflect their own style. The notion of more 

senior LMET graduates using LMET to reinforce past behavior, thereby 

reducing the likelihood of behavioral change is reflected in the 

following chief petty officer's comments: 

...not really, [no change] I found that what they 
brought out in LMET was basically what I was doing 
before I went. 

...you might be able to modify a man but you can't 
change his behavior. 

...I don't think so...[behavioral change] there are 
some things they pointed out that I do use...the way 
I handle the men is the same. 



was validated by supervisors and 


subordinates who observed the subjects before and after LMET school. 
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TABLE 7 

EXHIBITED BEHAVIORAL CHANGE OF LMET GRADUATES 


Yes Yes Yes/No* No No 

[A great deal [Minimum [Disagreement [little [Definatly 

of change] Change] between graduate change None] 

& supervisor/ if any] 
subordinate] 


Officer 

2 

5 

2/ 

3 

4 

CPO 

3 

2 

2/0 

0 

10 

P01 

6 

1 

7/1 

0 

3 

Total 

JJ[21.6%] 

8[15.7*] 

11/1[21.6*/1.9«] 3[5.9%] 

]7[33.3%] 


J9[37.3«] 

22[23.5«] 

20[39.2X] 


*In this column ''Yes H means the graduate states there has been a change and 
supervisor/subordinate state no change. A "No" means the graduate believes 
there has been no change while supervisor and subordinate say there has been 
a behavioral change. 









I can't say that [leadership style] has really 
changed...basically I am the same person, the way 
I lead my people. 

No, I think LMET made me realize some of my 
strengths and weaknesses so far as leading men, 
however, I thought I had more strengths than 
weaknesses. 

The more junior officers and petty officers seem to view 
LMET as a valuable and much needed training program. Among first class 
petty officers in particular, LMET training seems to provide the leader¬ 
ship/management theories and tools with which they are able to do their 
jobs. Additionally, this group has had sufficient experience to 
appreciate the curriculum structure and content. The data seems to 
indicate that first class petty officers are the most affected group of 
the 3 sampled levels and exhibit the greatest potential for behavioral 
change. For sampled junior officers, especially ensigns, there seems to 
be less indication of any behavioral change [for new ensigns this is 
primarily due to a lack of experience in not knowing how they led and 
managed others prior to LMET]. While these junior officers express a 
great felt need for the LMET training with its role in preparing them to 
assume their duties, their behavior does not indicate that they readily 
apply the competencies. Perhaps as was suggested by the junior officers 
themselves, LMET school [as presently scheduled], sandwiched between 
Surface Warfare Officer School and other specific training schools 
[Communication Officer, OCA School, Electronic Warfare, etc.], looses 
its impact by the time the officers reach the fleet. Then in the rush 
to fit into their operational command and specific billet the LMET 
training seems to get lost and forgotten, perhaps due to their inexperi¬ 
ence causing a lack of appreciation for the course content. 
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b. Weak Communications 


Of the five competency clusters taught during LMET school, 
two are particularly dependent upon good communication flow. The first 
cluster concerns the skillful use of influence and is paraphrased as: 
"Using influence in a positive fashion...not as a personal end-but 
towards Navy goals and effectiveness." [HRMC.N.D.] Components of this 
competency cluster include, influences, team builds, developes sub¬ 
ordinates and self-control. The thread that runs throughout this 
competency is communication. The analysis of data gathered during the 
study [Table 8] seems to indicate that sampled LMET graduates do not 
apply this competency to its fullest extent due to poor communications. 

A lack of communication, specifically in feedback concern¬ 
ing performance, was sited throughout the interviews of both graduates 
and their subordinates and supervisors. Interestingly, it included both 
the lack of positive as well as negative feedback, but especially 
emphasized the notion of positive feedback. This lack of positive feed¬ 
back among those interviewed indicated that a substitute for positive 
feedback became the very lack of any feedback. When querried as to how 
they knew when they were doing a good job, a common response from 
sampled graduates and subordinates was: "I really don't know, if I don't 
hear anything then I must be doing all right." This substitution of a 
lack of any communication for actual positive feedback appeared to be a 
common occurence among respondents. 

Other indications of poor communication involved a lack of 
including subordinates in the planning stages of task accomplishment 
[Table 8]. When the subordinate's opinions were solicited they were 
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TABLE 8 

LMET GRADUATE SELF-REPORT ON INFLUENCING 


This data was gathered during the course of answering three related 
questions concerning the process used to accomplish a newly assigned 
task, assigning duties and responsibilities to subordinates and the 
handling of roadblocks. Respondents were encouraged to site specific 
examples. The analysis rated the answers on a scale of 1-5 within the 
four relevant competency clusters. 


Officer LMET Graduates 



Low or No Use 
of LMET Compentency 

1 2 

3 

High Use of Lf 
Competency 
4 5 

Efficiency & 
Effectiveness 

0 7 

4 

8 

3 

Skillful Use of 

Influence 

1 2 

5 

4 

4 

Management Control 

0 0 

2 

6 

8 

Conceptual Thinking 

0 1 

3 

3 

9 

OVERALL: 

1 4 

14 

21 

24 
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TABLE 8 [con't] 


Low use of 

Chief Petty Officer LMET Competency 

LMt'T Graduate 1 

2 


3 

High use of 
LMET Competency 
4 5 

Efficiency & Effectiveness 

0 

0 


2 

3 12 


Skillful use of Influence 

0 

2 


3 

5 7 


Management Control 

0 

2 


0 

5 10 


Conceptual Thinking 

0 

2 


0 

6 9 


OVERALL: 

0 

6 


5 

19 38 


First Class Petty Officer 
LMET Graduate 

1 

2 


3 

4 5 


Efficiency & Effectiveness 

0 

1 


2 

6 9 


Skillful use of Influence 

1 

4 


2 

4 7 


Management Control 

0 

1 


2 

3 12 


Conceptual Thinking 

0 

2 


3 

5 9 


OVERALL: 

1 

8 


9 

18 36 


Additional data on the "Skillful Use of Influence" competency cluster 
was gained by asking the graduate, "How do you go about getting someone to 
buy your ideas about a better way of doing things?" 


Officer 

Chief Petty 
Officer 

First Class 
Petty Officer 

TOTAL 

Do nothing unless asked 

0 


0 


2 

2 

Order it done 

3 


1 


2 

6 

Argue that is the way it has 
always been done or you have 
seen it done this way before 
If accepted fine. 

1 


7 


4 

12 

Trial and Error Approach 

2 


0 


4 

6 

Argue strongly in its favor, 
pointing out strengths, make 
stand on its own merit. 

it 

10 


9 


6 

25 
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often rejected with little to no explanation, or with out trying them 
out. A common practice seemed to be not consulting with subordinates at 
all. 

"Advising and Counseling" is the second competency cluster 
affected by a weakness in communications and includes the components; 
positive expectations, realistic expectations and understanding. This 
competency group places particular emphasis on the communication aspect 
under the component of expectations. Realistic expections include: 

♦A realistic concern that instructions will be 
followed or carried out effectively by others. 

♦Open acknowledgement of negative expectations 
about the shortcomings of others. 

♦A willingness to express displeasure, dis¬ 
appointment, and concern about the short¬ 
comings of an individual's performance. 

[HRMO,N.O.] 

Again, communication is seen as an integral part of this competency. 

The interviews seem to indicate that in this area there is a great deal 
of communication between graduates and their subordinates. However, the 
emphasis appears to be only on the negative aspect while ignoring the 
other components of positive expectations and understanding. As Table 9 
indicates, supervisors are quick to provide verbal feedback and 
counseling for poor performance. Yet for good performance, none of the 
sampled LMET graduates indicated they would specifically counsel their 
subordinates. However, verbal feedback in the form of a "job well done" 
was given. The indication seems to be that the sampled LMET graduates 
view counseling as a tool for correcting poor performance with much less 
emphasis on good performance. 
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c. Rewarding and Disciplining Performance 

The fourth competency cluster taught during LMET school is 
that of "Management Control", which includes the following six 
components: 

♦Plans and Organizes 
♦Optimizes use of resources 
♦Delegates 
♦Monitors results 
♦Rewards 

♦Disciplines [HRMC,N.D.] 

The first four components of this competency appears to be 

used with varying degrees of success as indicated by the data [Table 8]. 

While sampled LMET graduates did not always indicate they involved 

others in the planning, they certainly made efforts to optimize 

resources and delegate tasks to a lower level. The last two components, 

rewards and disciplines, did not appear to be integrated into the 

sampled LMET graduates behavior. 

As indicated in Table 9, the use of rewards was 

substantially limited to three areas, verbal recognition [formal and 

informal], time off from work, and written evaluations. Of these three 

responses, written evaluations may be downplayed, as few respondents 

suggested special or commendatory evaluations but rather indicated good 

performance would be noted on formal, periodic written evaluations. 

Since the Navy evaluation process expects or insists supervisors do 

this, it cannot be truly viewed as much as a reward as something earned. 

This feeling was best expressed by LMET subordinates, one who remarked, 

"He didn't give me a good evaluation, I earned it!" The other two 

major types of rewards were verbal rewards and time off from work, with 

both more heavily relied upon by the first class petty officers. 

Again, these rewards have come to be so 
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TABLE 9 

LMET GRADUATES SELF-REPORT ON ADVISING AND COUNSELING 
AND REWARDS AND DISCIPLINES 


This data was gathered from responses to two related questons, 
"What do you do if the work of a subordinate is: worse than expected? 


Better than expected?"* 






No or Low use 
of LMET 
Competencies 

Officer 1 2 

3 

4 

High use of 
LMET Competencies 
5 

Efficiency & Effectiveness 

1 

1 

3 

5 

6 

Skillful use of Influence 

0 

2 

6 

2 

6 

Advising and Counseling 

3 

0 

1 

3 

9 

Management Control 

0 

3 

3 

3 

7 

OVERALL: 

4 

6 

13 

13 

28 

Chief Petty Officer _ 

1 

2 

3 

4 

5 

Efficiency & Effectiveness 

0 

0 

1 

8 

8 

Skillful use of Influence 

0 

1 

5 

5 

6 

Advising and Counseling 

0 

1 

1 

6 

9 

Management Control 

0 

1 

4 

7 

5 

OVERALL: 

0 

3 

n 

26 

28 


*This Table was prepared using the LMET graduate's responses from the 
interview tapes. The scale was developed as a convenient method of 
presenting the data. 












TABLE 

9 [con't] 




First Class Petty Officer 

1 

2 3 

4 

5 


Efficiency & Effectiveness 

0 

0 1 

8 

9 


Skillful use of 

Influence 

0 

1 7 

3 

7 


Advising and Counseling 

1 

1 0 

7 

9 


Management Control 

0 

1 6 

5 

6 


OVERALL: 


1 

3 14 

23 

31 


Specific responses to the two querries follow: 




"If work is worse" 





• 

Counsel 

Verbal 

Reprimand 

Re-Oo Extra-Military Formal Reassign 

Task Instructional] Punishment Task 


Officer 13 

8 

3 

2 

3 

2 


CPO 15 

12 

6 

0 

3 

0 

r , 

S 

P01 15 

10 

5 

5 

4 

0 


"If work is better" 





( 

Verbal 

Reward 

Physical/Material 
Award [Buy a coke, 
handshake, pat on 
back, etc.] 

Time-Off 

Written 

Evaluation Formal Award 

Recommenda- Commendation 

tion for 

advancement 

% 

Officer 11 

4 


10 

6 

6 


CPO 15 

4 


10 

9 

8 

■ < 

t ’ 

P01 16 

6 


16 

11 

4 
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institutionalized as to almost lose their status as rewards. Personnel 
who work late, especially hard or do a job particularly well expect to 
be rewarded both verbally and with time off. To not give these rewards 
seems to be viewed as a form of punishment. In addition, all three of 
these rewards require only a small amount of effort for the supervisor, 
hence, the propensity towards their use. The fact that chief petty 
officers and officers are reluctant to give time off as compared to 
petty officers may be indicative of their view of the organization, the 
amount of work to be completed as well as more direct pressure from 
seniors to account for absent personnel. Written awards, commendations 
and recommendations for medals was given a lower priority as a means of 
rewarding personnel for good performance for a variety of reasons. Most 
common seemed to be the concept that good performance was expected, 
hence not worthy of such a "high" reward. Yet few graduates were able 
to verbalize what sort of performance did deserve such an award. It was 
the interviewer's perception that the problem was not the level of the 
award but the amount of personal involvement needed to give such an 
award. 

The last component, discipline, seemed to share the same 
sort of priorities as rewards. That is, disciplinary actions that 
involved a substantial amount of interaction with the supervisor were 
avoided. As Table 9 indicates, counseling [negative feedback] and 
verbal reprimands were the most commonly cited means of disciplining. 
It Is perhaps no coincidence that these forms involved the least amount 
of a supervisor's effort. Even "counseling" can be conducted with 
little preparation and informally so as to be very efficient from a time 
perspective. Redoing the task and Extra Military Instruction [EMI] both 









involve a high degree of interaction with the supervisor in the form of 
close monitoring and instruction. Both methods were shunned as 
indicated by the data, especially at the officer/chief petty officer 
level. Actual punitive measures such as non-judicial Captain's Mast 
were viewed by those respondents who mentioned it as a last ditch 
effort, only to be used when all else failed. What is significant is 
the relatively few respondents that even mentioned it. A final note 
should be made concerning the officers who would reassign a task 
improperly completed to another person who could be relied upon to get 
it completed. This would seem to indicate an extreme propensity for 
task accomplishment at any cost, and if so, would be highly divergent to 
the LMET approach to leadership. 

d. Increased Awareness 

A final area of behavioral change concerns overall change, 
perhaps better called an awareness or attitudinal change. That is, the 
graduate's awareness towards, leadership/management as well as his 
affect on other people has increased which is manifested in some 
behavioral changes. Table 7 indicates that approximately two thirds 
[6156] of the sampled graduates expressed either a change in their 
leadership style or in adopting a leadership philosophy more closely in 
line with LMET. Of these graduates, half have in fact changed or 
adopted an LMET approach as validated by their supervisors and 
subordinates. One third of the total sample have exhibited no change as 
validated by their supervisors and subordinates. This means about one 
fourth [23.556] of all graduates believe they have changed yet this 
change is not validated. The significance of this data seems to be that 

behavioral change brought about by LMET varies greatly, based upon a 
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variety of factors. However, it does seem clear that behavioral change 
as a specific result of LMET school is not at all assured. 

4. Results 

The last domain or level to be examined is that of results, 
which in fact is what an evaluation of the entire LMET Program should 
most likely explore. The priority question is whether the program does 
in fact result in real changes to the bottom line measures. This pilot 
study was not designed to definitively address the specific domain of 
"results". Without a control group of non-LMET graduates to compare 
against, and the numerous non-LMET related factors that affect 
productivity [e.g., technical training, size of work group, imagination, 
etc.] there may be no satisfactory method of determining the cause and 
effect relationship between LMET and "results" outcomes. Despite these 
difficulties and recognizing the limitations of the sample itself, some 
indications of the effect of LMET on bottom line measurements were 
uncovered. 

a. Job Performance Changes 

First, most sampled LMET graduates reported that job per¬ 
formance measures had increased or remained high in their work groups 
[this was validated by both subordinates and supervisors] since their 
completion of training [see Table 10]. Of particular interest is that 
first class petty officers report lower perceived impact on work group 
performance than do the chief petty officers or officers. This may 
reflect the first class petty officer's perception that being lower in 
the chain of command affords them less chance to influence the system. 
The relatively high percentage of officers [63<] that saw an Increase 
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TABLE 10 


CHANGE IN THE LMET GRADUATE'S WORK GROUP PERFORMANCE AND SATISFACTION 


Perceived Performance Changes: 


Yes, 

Yes, 

No, 

No 

No, 

Yes, 

Yes, 

major 

minimum 

remained 


remained 

Minor 

major 

increase 

increase 

high 


low 

decrease 

decrease 

Officer 6 

4 

1 

3 

0 

1 

1 

CPO 4 

7 

3 

3 

0 

0 

0 

P01 4 

6 

1 

6 

1 

0 

0 








OVERALL: 14 17 5 

Perceived Satisfaction Changes 

12 

1 

1 

1 

Officer 7 

2 

0 

5 

0 

1 

1 

CPO 1 

7 

4 

2 

3 

0 

0 

P01 3 

7 

1 

6 

1 

0 

0 




* 




OVERALL: 11 

16 

5 

13 

4 

1 

1 











may well reflect a willingness on their part to experiment with 
leadership/managerial styles as well as use of their relative position 
of higher influence. Chief petty officers may also enjoy a position 

form which it is easier to influence improved performance. This 
hypothesis is supported by a comment from one subordinate who stated, 
"We got a chief so things got better." 
b. Job Satisfaction Changes 

As for satisfaction measures within a work group [increased 
retention, less non-judicial punishments, less sick calls and 
absenteeism], there was no consistent pattern of change across the 
entire sample which could be attributed to LMET. While Table 10 seems 
to indicate some positive changes in satisfation, the respondents 
credited most change to environmental changes [e.g., return from 
deployment, finishing shipyard overhaul, etc.] or to the personality 
[e.g., nice person, good conversationist, easy going, etc.] of the LMET 
graduate and not so much to the leadership style. However, the inter¬ 
views seemed to indicate that when LMET graduates did infact involve 
subordinates in the decision making process and opened clear lines of 
communication that job satisfaction did increase. Of particular note 
was the case of two petty officers who did a complete turnaround in 
leadership/management styles after LMET [attended on a TAD basis]. In 
their cases the change was clearly noticable to supervisors and 
subordinates and in fact resulted in measureable increases in both job 
performance and satisfaction measures. 







c. Retention Aspects of LMET 

Concerning retention, the data do not indicate any 

systematic cause and effect relationships between retention and LMET 
training. However, there were specific cases in which there appeared to 
be an LMET influence toward increased retention within the work group. 
The graduates themselves indicated that LMET training had little effect 
on their own career plans. This is perhaps not surprising when 
examining who is actually attending LMET school. Chief petty officers 
and first class petty officers tend to be career oriented personnel or 
have at least made definate plans concerning re-enlistment. Regardless, 
they tend to be a tough group to sway one direction or the other. 

Junior officers on the other hand, while more undecided about their 
careers, appear to-be unwilling to make a career committment based soley 
upon a positive LMET experience. 

LMET graduates on the other .hand, did have some positive 
effect on retention within their work groups. Several subordinates 
readily admitted that the actions of their LMET graduate boss had in 
fact been the determining factor in their re-enlistment decision. One 
LMET graduate had a 100 per cent re-enlistment rate [supervisor-report¬ 
ed]. However, the question still remains as to whether this was all or 
even in part due to LMET. At least one certainty can be stated, that is 

that no respondent indicated they were leaving the Navy as a result of 

LMET. 

d. Developing Healthy Climates 

With respect to developing healthy organizations or work 
groups, LMET graduates seemed to give this a high priority and are in 
fact successful as indicated In Table 11. The majority of the sampled 
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TABLE 11 


INDICATION OF HEALTHY WORK GROUPS OF LMET GRADUATES 


This data was gathered on the question, “What happens to your work 


center when 

you are not 

there?“ 





1 

Things fall 
apart 

2 

Just enough 
Is done to 
stay out of 
trouble 

3 

Major jobs 
continue, 
some minor 
ones don't 

4 

All work 
done, accept 
supervisors 

5 

All work 
gets done 
including 
supervisor 

Officer* 

1 

1 

4 

7 

2 

CPO 

1 

1 

7 

3** 

5 

POl*** 

2 

0 

4 

3 

9 

OVERALL: 

3 

2 

15 

13 

16 


♦One officer was not asked this question as he was in the process of 
relieving the old division officer. 

♦♦These chief petty officers all specified that only the division officers 
collateral duties would not get done. 

♦♦♦One POl was not asked this question due to his newness to the work group. 









LMET graduates stated that they had in fact trained their subordinates 
to take over in their absence. There was some question as to how well 
their work group would function [quality of performance], but not 
whether it would function. In those few instances when the LMET 
graduate indicated that the work group would not function at all or at a 
level to barely get by, the LMET graduate appeared to not practice the 
LMET competencies, especially skillful use of influence. The overall 
indications were that LMET graduates who practiced what was taught, 
tended to create healthy organizations by training subordinates to take 
on increased responsibility, delegating decisions to lowest possible 
levels, encouraging communication up and down the chain of command and 
using their influence to positively motivate their subordinates, 
e. Handling Newly Reported Personnel 

A related area to retention and building a healthy work 
group is how new personnel are integrated into the work group. Table 12 
indicates that LMET graduates tend to concern themselves with a new 
persons first impression. Most graduates stressed the use of the 
reporting aboard interview as a means of explaining where the new person 
fits within the organization as well as his importance to the 
organization. The interview was also used to inform the new individual 
of promotion plans, career development, training programs, and daily 
routines/standard operating procedures. This interview stressed the 
necessity of discussing any personal problems that might prevent easy 
transition into the work group. The one subject that was not discussed 
tended to be the expected performance or goals for the newly reported 
person. Most respondents stated it was expected that this information 






TABLE 12 


LMET GRADUATE SELF-REPORT ON HANDLING NEWLY REPORTED PERSONNEL 


1 2 3 4 5 

[Nothing done [Administrative [Full welcome 


Officer 1 

personally] 

0 

3 

business only] 

4 

3 

aboard brief & 
interview] 

4 2 

CPO 

1 

1 

3 

7 3 

5 

P01 

2 4 

1 

6 5 

2 

7 6 







OVERALL: 

3 

5 

13 

12 

16 


iTwo officers not asked this question 

2"_Royal Red Carpet Treatment." - Division Officer 

"....Like to find out what they expect from the Navy to help satisfy 
their need." - Division Officer 

3"I don't want him to do the shit jobs right off the bat." - CPO 
"I let him play pigeon for a few days.” - CPO 

4"I just put them straight to work....the first day they come to me, I 
give them a chipping hammer and a wire brush and away they go." - P01 

5"I see all personnel for administrative purpose only." - P01 

6"_I guess I don't really tell him what I expect...I don't express it.. 

I assume he will talk to his co-workers." - P01 
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would be picked up from peers. This idea was best stated by an LMET 

first class petty officer, "I guess I don't really tell him what I 

expect...I don't express it...I assume he will talk to his co-workers." 

Aside from the reporting aboard interview, LMET graduates 

expressed a wide variety of methods for handling newly reported 

personnel as indicated in the following interview remarks: 

I don't want him to do the shit jobs right off 
the bat...CPO 

...let him play pigeon [running-mate concept] for 
a few days - P01 

...I like to find out what he expects from the 
Navy to help satisfy his needs - Div Off 

...I just put him straight to work...The first day 
they come to me I give them a chipping hammer and 
a wire brush and away they go - P01 

...I give them the royal red carpet treatment for 
a few days - Div Off 

The LMET graduates that tended to place less emphasis on smoothly 
integrating personnel into their work groups more consistently were 
those that had the least healthy organizations, 
f. LMET Reinforcement 

Another aspect of the "results" level was briefly 
discussed earlier within the knowledge-content domain. It concerns the 
LMET graduates unmindful disregard of LMET concepts and vocabulary at 
the beginning of the individual interviews with a noticeable improve¬ 
ment as the interview progressed. It appeared that the graduate had 
totally forgotten some of the LMET issues and language of the training. 
As Table 13 indicates, 61% of the graduates never used their student 
journal after completing the LMET training. More significantly, only 
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TABLE 13 


LMET GRADUATE'S SELF-REPORT ON USE OF THE STUOENT JOURNAL 


The initial question for this data was, "Have you ever used the 
student journal." In some cases the response was "Yes" but not for 
reasons dealing with the LMET competencies. Yes answers reflect both 
aspects. 



TES 

NO 

Officer 

7 1 

9 

Chief Petty Officer 

5 2 

12 

First Class Petty Officer 

8 3 

JO 

TOTALS [<]: 

20 [39] 

31 [61] 


*0f these seven, 4 used it for reasons related to LMET. 
2 Of these five, 3 used it for reasons related to LMET. 
30f these eight, 7 used it for reasons related to LMET. 
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27 % of the sample actually used the journal as a reference or guide to 
a problem they were encountering. This is inspite the fact that the 
manual's intended use is as a guide and reference list for the LMET 
competencies. Additionally, it is designed to be used in helping to 
establish and track self,improvement goals in leadership and management 
development for each graduate. The indication seems to be that by not 
actively using the student journal, LMET graduates are not keeping the 
compentencies foremost in mind as tools for leadership and management. 

A last general observation, based on responses to the 
query as to whether they had anything else they would like to mention, 
was that if a command explicitly or implicitly discouraged the use of 
the LMET situational approach to leadership and management, a very high 
level of frustration developed within the graduate. Graduates that 
expressed having experienced this problem seemed to use three different 
techniques to cope with their anxiety. First, there were those who 
continued as taught through LMET and just lived with the frustration. 
This, of course, led to high dissatisfaction towards their job and 
sometimes created conflicts within their work groups. The second 
approach seemed to be that of doing the exact opposite of what LMET 
taught. For example, using the authoritarian approach to its extreme. 
Interestingly, these graduates viewed themselves as using the "old 
Navy" approach and felt they were completely in line with recent CNO 
policy statements concerning "pride and professionalism". The third 
method seemed to be the development of a more passive existence through 
the removal of oneself from the active chain of command, becoming more 
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of a bystander [especially in one instance with an LMET chief petty 
officer]. Apparently the graduate felt a loyalty towards the LMET 
training but could not handle a situation when the supervisor did not 
exhibit any support. Again, these were exceptions to most graduates 
experiences as expressed in the interviews, but may nonetheless 
indicate a strong need for external support mechanisms in order for 
LMET to succeed. 

C. SUMMARY OF LMET FINDINGS 

Again, the findings concerning LMET must be strictly viewed within 
the context of the limitations of a non-random sample. The issues 
Identified and the points raised should be considered hypotheses which 
need to be resolved through either further study, or within a more 
complete full-scale evaluation to test their actual validity. However, 
the following general comments, while not all inclusive, are felt to be 
Indicative of the pilot study sample and are therefore included as a 
convenient summary. 

Reactions of personnel interviewed concerning LMET were upbeat, 
with the majority of respondents in agreement as to its necessity and 
usefulness. LMET was viewed as a positive experience which seems to 
have affected graduates in differing degrees. The actual knowledge of 
LMET training, while not always initially evident, appeared good. 
While systematic behavioral changes linked directly to LMET were not 
uncovered, there were specific examples and indications that behavioral 
changes had taken place. 
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LMET program strengths such as graduate increased awareness, the 
fulfillment of manager's training needs in the areas of management/ 

leadership, developing a positive framework for healthy work groups, 

among others were notably in evidence. Several areas of concern were 

identified and require further investigation to determine their 

validity. Among these issues were the following four: 

Communication Aspects of Rewarding and Disciplining 
The Setting of Goals and Performance Standards 
Feedback Concerning Performance 
The LMET Student Journal 

It is unclear to the extent in which the LMET pilot study findings 
are generalizable to the entire population. They nevertheless re. 
present new avenues to explore in developing a comprehensive evaluation 
plan for LMET. 
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VI. RECOMMENDATIONS AND CONCLUSIONS 


This final chapter contains the conclusions concerning the useful¬ 
ness of the pilot study concept as well as recommendations concerning a 
future overall evaluation of the LMET program. As such, the chapter 
emphasizes an overview of three general areas of concern; the pilot 
study concept, particularly as applied to the evaluation of LMET; the 
interview as a data gathering methodology; and relevant issues 
requiring further investigation in developing an evaluation plan for 
the Navy's LMET program. These three areas of concern are divided into 
two sections for presentation; first, the pilot study conclusions and 
second, the recommendations concerning an LMET evaluation. 

A. PILOT STUDY CONCLUSIONS 

The central theme throughout this research has been the pilot 
study concept. Although previously stated, it is of sufficient import¬ 
ance to re-emphasize that the data sample for this study was neither 
random nor representative of the Navy's LMET graduates. The 51 LMET 
graduates that were interviewed contained a cross-section of various 
operational commands, billets, rates, ranks, warfare specialties and 
geographic locations from the west coast. However, the data was suffi¬ 
cient for this pilot study in attempting to identify issues, test 
hypotheses, and evaluate a methodology pertaining to the future overall 
evaluation of a major Navy program such as LMET. In no way has this 
pilot study eliminated the need for an LMET evaluation, rather it rein¬ 
forces that need. 
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An important question that remains unanswered and requires further 
examination is, how should effectiveness of LMET or any similarly large 
program be measured? This pilot study concentrated on individual 
behavioral changes in leadership/management after LMET graduation. 
Other effectiveness criteria for the LMET program could include, but 
are not limited to: individual command effectiveness, reduction of 
selected Navy-wide problems or concerns [e.g. retention, substance 
abuse, etc.], and knowledge/behavior of graduates 6 months after LMET 
training. The underlying issue is determining what is the actual 
purpose or goal of the Navy's LMET program. Once that is dearly 
defined, then an evaluation of its progress towards achieving that goal 
can more easily be conducted. 

The continued emphasis on the limitations of this pilot study is 
not intended to diminish its usefulness to LMET program managers and 
decision makers. Rather, it is to keep the pilot study results within 
their proper perspective, that of providing preliminary guidance 
towards a full-scale LMET evaluation. This study did examine the LMET 
training with the stated objective of providing substantive recommenda¬ 
tions for conducting a full-scale evaluation of the program. The 
beneficial accomplishments derived from the pilot study concept as re¬ 
lated to general program evaluation fall into four categories; 
efficiency, effectiveness, definative results, and participant educa¬ 
tion and learning. 
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1. Efficiency 

The first accomplishment, that of efficiency, can be ex¬ 
amined from three different resource perspectives: minimum manpower, 
low cost, and short duration. This pilot study was conducted by two 
lieutenant commander. Naval Postgraduate School students in working on 
their master's degree thesis. Their time was already committed to some 
form of thesis work, and in undertaking a pilot study of the Navy's 
LMET program, they, as well as the Navy, were able to benefit in 
"killing two birds with one stone". While it may not always be easy to 
find such an efficient situation, the fact of the matter is that two 
Navy personnel conducted the study, which underscores the efficiency of 
this approach. Further emphasizing efficiency is the pilot study's re¬ 
latively low cost. Only $10,000 was allocated for this research, 
which, when compared to an overall program evaluation, seems well 
worthwhile to insure the usefulness of the full evaluation. Finally, 
there is the timeliness aspect of the pilot study, 6 months from start 
to finish. It should be noted that this was accomplished in addition 
to regular ongoing classes and studies. 

2. Effectiveness 

The next accomplishment of the pilot study is in the area of 
effectiveness. When this pilot study commenced, the problem that was 
most difficult to acertain was how to measure effectiveness of such a 
large program. It was not clear at the outset exactly what to measure 
or how. The flexibility afforded by only two people 
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undertaking such a study was another primary cost savings. As issues 
were identified, questions developed, and the evaluation method 
formalized, the two researchers were able to easily adjust and proceed 
with few problems. This very flexibility permitted the gathering of an 
enormous amount of high quality information on a complex, dynamic Navy¬ 
wide system. 

3. Definitive Results 

The third major accomplishment of this pilot study is that it 
produced definitive results. It did develop issues and recommendations 
on how to conduct a full-scale evaluation of the LMET program. These 
findings, issues, and recommendations can be used by program managers 
and decision makers in developing a plan for the further evaluation of 
this program with a certain amount of assurance that they are indeed 
proceeding in a relevant direction. 

4. Education and Learning 

The final accomplishment, that of education and learning, 
while not an intended goal, further establishes the value of a pilot 
study. In order to develop a pilot study design for the LMET program, 
it was necessary to first learn about the program. The background re¬ 
search conducted resulted in insights into the overall LMET program, 
from theoretical conception to development through implementation. 
Additionally, a feeling for and an understanding of the program, as it 
is now administered, was gained by the researchers. The implication is 
that these two researchers may be as knowledgeable, if not more so, 
than the actual program managers of LMET. This educational feature of 
the pilot study process is a prime argument for program managers to 
become Intimately involved. 
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5. Future Pilot Studies 


With these four considerations in mind let us now briefly 
examine the future potential of pilot studies within the Navy. On the 
surface it appears that the benefits far outweigh the costs of a pilot 
study completed prior to a comprehensive program evaluation. If more 
program evaluations were designed with a pilot study as their first 
step, program managers by involving themselves, could feel more confid¬ 
ent in directing the full“scale evaluation. A pilot study can examine 
relevant issues such as who, how and what to evalute. Program 
managers using this approach can gain valuable insight and direction 
for their program evaluations, thereby making better decisions. The 
pilot study concept can be viewed as another tool that managers and 
decision makers can use to improve the quality and timeliness of their 
decisions. 

8. RECOMMENDATIONS CONCERNING LMET EVALUATION 

This section addresses some definitive recommendations concerning 
a future full-scale evaluation of LMET. These recommendations are 
organized to answer four central questions concerning that evaluation: 
who?, where?, how?, and what? While not all inclusive, it is felt that 
these insights will provide sufficient guidance to LMET program mana¬ 
gers to develop a useful evaluation plan. 

1. Who 

The first question concerning the evaluation plan is who, and 

can be further divided into four topical areas; the individual LMET 
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graduate, the chain of command, operational commands, and evaluators. 
All of these areas when viewed together shed light on the who question, 
a. LMET Graduates 

In order to evaluate the effectiveness of the LMET program 
it is necessary to collect data from those individuals most affected by 
the training, which in this case are the graduates. Behavior modifi¬ 
cation or improvement in the area of leadership/management is one of 
the goals of the program. In order to collect relevant data which 
reflects the extent of this improvement [as well as a possible cause 
and effect relation to the training], a sufficient amount of time after 
graduation must have elapsed in order for the graduate to have develop¬ 
ed his operating style. It is recommended that data be collected from 
LMET graduates who have been out of LMET school for a minimum of 6 
months. However, as time increases from graduation, a cause and effect 
relationship may become contaminated due to other factors. In addi¬ 
tion, due to the multi-levels of training as well as the number of 
graduates, data collection should concentrate specifically on middle 
managers, such as: department heads, division officers, chief petty 
officers [CPO] and first class petty officers [P01]. In comparing the 
responses of department heads/CPO's with division officers/POl’s the 
issue of seniority as compared to changes in behavior can be further 
explored. Realizing that data collected from graduates [self-report] 
may be suspect, it is further recommended that a validation procedure 
be implemented. The validation process used in the pilot study of 
collecting additional data on the graduate from his immediate super¬ 
visor and subordinate would work quite well. 
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b. Chain of Command 

An issue that surfaced as a result of the pilot study, was 
the effect that the chain of command had on a graduate towards applying 
the training. By collecting data on the entire chain of command, the 
effect of command support for the graduate can be further explored. 
Additionally, as time permits, indications of whether the graduate is 
considered an average or superior performer should be sought to assist 
in confirming the graduates self-reported information, as well as a 
first step in revalidating the competencies on a hierarchical basis. 

c. Operational Commands 

Since the LMET program has been initially directed at 
operational commands, the greatest impact, hence potential for change, 
would be expected to occur there. As such, operational commands should 
be the specific targets of the evaluation. One additional considera¬ 
tion is in the choice of which graduates to specifically emphasize. 
Since leadership/management behavior [change or improvement] is the 
emphasis of the evaluation, only graduates in this role should be con¬ 
sidered. Air squadrons for example, have many junior officer [ENS, 
LTjg] graduates who do not hold active leadership/management positions 
and would therefore provide little substantive data. 

d. Evaluators 

The final issue is who should conduct the evaluation - 
civilians or Navy personnel? Constraints of sample size, time and man¬ 
power availability may be the deciding factors of actually who will 
conduct the evaluation, civilian or military personnel. However, one 
issue is clear, that the program managers should take an active role 
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in order to educate themselves on the LMET program. The issue of 
whether to employ civilian or military evaluators hinges to a great 
extent on the data collection method used. If interviewing. Navy 
personnel may have an edge, as insiders to the organization. However, 
properly trained civilians, skilled in areas of behavioral psycology, 
may be able to quickly overcome that advantage due to their insights on 
human behavior. Additionally, Navy personnel may be more willing to 
express negative opinions on the LMET program to civilians. Obviously, 
using both types of personnel, exploiting their strengths would most 
likely provide "the best of both worlds". 

2. Where 

The next major question to be addressed is where to conduct 
the evaluation. To reinforce an issue discussed earlier, the 
evaluation should not be conducted at the LMET school. Here graduates 
will most likely provide insights of their knowledge and not behavior. 
Onboard individual commands appears to be the most ideal location to 
gather behavioral information. In this way the respondents can remain 
within familiar, more comfortable [psychologically] surroundings. It 
is also less disruptive to individual command's work schedules, hence 
more likely to gain their acceptance and support. The specific concern 
is that it be conducted in a neutral zone within the command where the 
respondent will be less pressured by the internal environment. One 
issue that did surface during the pilot study is that of the command's 
environment. A cross-section of environments should be included with¬ 
in the evaluation sample. There was an effort during the pilot study 
to concentrate on commands with easy access [i.e. inport within the 3 
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geographical areas], hence, a tendency to evaluate graduates on 
commands in the shipyards, undergoing heavy inport maintenance programs 
or training programs. Commands in all phases of employment should be 
considered such as in shipyards, in homeport, deployed, during overseas 
work.up exercises, and on overseas standdowns, so as to not bias the 
data due to any unknown external environmental effects. 

3. How 

The third question, that of how, concerns the methodology to 
use in conducting the evaluation. This section is further divided into 
three areas of concern which are: use of interviews, use of surveys, 
and mechanics of interviewing. 

a. Use of Interviews 

The pilot study used interviews as the primary data 
collection method. It was found to be an effective means of gathering 
data concerning non.quantifiable, behavioral.oriented responses. The 
LMET Stimulus Questions provided by McBer and Company were particularly 
useful in exploring these areas. When the desired data is directed to. 
wards actual behavior with the multitude and variety of responses 
possible, interviewing seems to be the best method. While straight 
forward questions seeking limited responses, factual and quantifiable 
data are more efficiently collected using methods other than 
interviews. 

b. Use of Surveys 

Surveys, as opposed to interviews, are better suited 
toward easily quantifiable reponses, factual data, and limited 
responses. Straight forward questions that are less affected by the 
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respondents personal opinions a~.d more directed at specific answers and 
ranges of occurrences are more efficiently contained in a survey. The 
survey can encompass a larger sampie size quickly within the 
limitations previously disscussed in earlier chapters. McBer's 
"Management Behavior Survey" [partially contained in Table 14] is an 
example of how to gather data on the graduate's usage of LMET 
competencies. When asking straight forward questions such as these on 
the competencies, the resulting data is sufficient for a broad general 
overview of implications. If such a survey is administered to only the 
graduates, it takes on the characteristics of an unvalidated 
self-report and may therefore indicate knowledge and opinion rather 
than actual behavior. Through a validation process with supervisor and 
subordinate input, such a survey could provide insights into a 
graduates behavior. 

c. Mechanics of Interviewing 

Since interviewing was the primary data gathering method 
for this pilot study, it is appropriate to mention some of the lessons 
learned to assist in using this method within a full-scale evaluation. 
The presentation is sub-divided into four general areas of: training, 
interview introduction, methodology and tools. 

First, the training necessary to successfully conduct in¬ 
terviews is minimal. While there are numerous references, R. L. 
Gorden 1 s book Interviewing: Strategy, Techniques and Tactics provides a 
useful background on interviewing. A seminar or workshop on inter¬ 
viewing as applied to the LMET evaluation process should provide 
interviewers with sufficient skills to grasp the basic techniques. With 
additional practice and critiquing most individuals should be able to 
develop into skilled Interviewers. 
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TABLE 14 


SAMPLE ITEMS FOR MANAGEMENT BEHAVIOR QUESTIONNAIRE 


Circle the number which best represents where the ratee's behavior 
falls relative to the two statements provided. [For example, circling 
2 would indicate a greater tendency to behave in accordance with the 
statement on the left; circling ± would indicate that both statements 
are equally descriptive of his or her behavior.] 


Sets goals and performance standards: 


1. He/She sets clear specific 
goals. 

1 2 3 

2. He/She will change the goals 
to make them realistic. 


1 2 3 

3. He/She sets milestones when 
assigning a task and checks 
to see if things are on 
schedule. 


Takes Initiative: 

4. He/She initiates action 
rather than waiting to 
react as situations develop. 

1 2 3 

5. He/She experiments with 
and introduces innovative 
work procedures. 

1 2 3 

6. He/She spends unusually long 
hours on tasks when they 
require it, whether, he/she 
is asked to do so or not. 

1 2 3 


He/She sets vague goals. 


4 5 6 7 

He/She will keep trying to 
reach a qoal even if it does 
not appear realistic. 

4 5 6 7 

He/She gives out a task and 
checks back about the time it 
should be done. 


4 5 6 7 


He/She takes appropriate 
actions to handle situations as 
they develop. 

4 5 6 7 

He/She uses traditional, well- 
established work procedures. 


4 5 6 7 

He/She will work regular hours 
on a task and if It takes 
longer than that to complete 
will come back to it later. 

4 5 6 7 
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Second, the introduction to the interview is a key to 
establishing good rapport with the respondent, thereby ensuring a good 
flow of information. A good introduction needs to stress the 
conf identiality of the interview and emphasize that its overall intent 
is to evaluate a program and not the individual. Putting the respond¬ 
ent at ease initially is important and can be assisted by beginning the 
interview with easier "warm-up" questions. A final point, is that most 
groups of related questions have a natural flow, which should be 
identified and followed. This will assist in guiding the interview 
process and in fact eliminate the need to ask certain questions as they 
may be answered before they arise. 

The next issue is that of the interview method, and 
specifically concerns interviewer/respondent interaction. The purpose 
of the interviewer is to guide the conversation in the desired 
direction without stifiling the information flow. Respondents will 
normally use the interview as a means to voice their opinion in 
unrelated areas, hence they may ramble or get off onto tangents. Firm 
direction from the interviewer can keep it both healthy and on track. 
However, one should not get so locked into a rigid process as to miss 
the surfacing of relevant issues. These areas should be explored fully 
to uncover any new areas of concern. 

The final area concerns the interviewing tools available. 
A variety of "tools" are available for conducting interviews, each with 
definate advantages and disadvantages. The final decision on using a 
tool is how it will assist in the interviewing process. If it does not 








provide advantages or specific assistance to the Interviewer its use 
should be carefully reconsidered. Some “tools" available for use by 
the interviewer include tape recorders [should have tape counter for 
indexing], coded answer sheets, and preformated interview schedules. 
This last item was found to be a valuable tool in conducting the pilot 
study, in that it assisted the interviewers in standardizing their 
daily routine while providing the command with clear guidance as to how 
to schedule their personnel. 

4. What 

The final question, that of what, concerns issues which should 
be considered in an evaluation of the LMET program. The pilot study 
surfaced certain issues which require further investigation in the 
course of a full-scale evaluation. These findings should be considered 
unvalidated hypotheses that require further study or additional testing 
within an overall evaluation process. There are seven major issues 
that will be discussed: graduate's knowledge level, increase in 
self-awareness, cause and effect relationships, communication skills, 
use of the journal, retention, and reinforcement of behavior. 

a. Level of LMET Graduate's Knowledge 

While not specifically the emphasis of the pilot study, 
certain levels of knowledge exhibited by LMET graduates became evident. 
Further evaluation of the LMET program should in fact try to determine 
the level or degree to which LMET knowledge is retained after gradua¬ 
tion by examining the extent to which their vocabulary and concepts 
conforms to the LMET competency based model. In addition, the ability 
to discuss techniques actually practiced can be compared with course 









instruction to provide some insight into how well the competency model 
is understood. This data may also provide insight as to the effective¬ 
ness of the classroom instruction at the school. 

* b. LMET Graduate Self-Awareness 

The pilot study data do indicate that graduates have a 
heightened sense of awareness on how their actions effect others as 
well as how the actions of others effect them. This issue deserves 
further investigation, in that this increased self-awareness could be 
the first step in a more definative behavior change in leadership and 
mangement style. While admittedly a self-report, if it is supported by 
indications of a behavioral change then LMET should be credited for 
beginning the trend towards modification of leadership/management 
skills towards the competency based model. 

c. Cause and Effect Relationships 

The pilot study data do not support any systematic cause 
and effect relationship between LMET graduates and increased 
performance. There were, however, at least two cases [validated by 
supervisor and subordinate] where the LMET training resulted in a 
complete and highly visible turnaround in a managers behavior and 
performance. However, both cases involved managers who were considered 
marginal performers and were sent to the school as a last improvement 
effort. The Important Issue that remains to be determined is: does 
LMET school improve leadership/management behaviors, hence Increase 
managerial performance? A side Issue seems clear, in that no cases 
were uncovered showing any decrease in performance after LMET school. 
There appears to be a definate need for further evaluation on this 







subject, to determine if LMET training alone can bring about managerial 
behavior change or improvement resulting in increased performance. If 
LMET school itself is not sufficient, what if any additional support is 
needed? 

d. LMET Graduate Communication Skills 

LMET graduates overall demonstrated a noticeable weakness 
in the use of communication skills and techniques in their leadership/ 
management roles, as supported by the pilot study data. The issue of 
communication skills needs to be further investigated in order to 
identify specific areas that may need additional emphasis during the 
LMET training. Of the sixteen LMET taught competencies, four were 
identified as specific concerns from the data. These were: a lack 
among LMET graduates in setting performance standards; a lack of goal 
setting, both personal and workgroup; limited use of rewards and 
disciplines, coupled with a lack of flexibility for different 
situations; and the use of advising and counseling for correcting poor 
performance only. Each of these specific competencies can be treated 
as a seperate issue for further examination and evaluation. However, 
they should be recognized as related by a common thread as forms of 
managerial communication. 

The perception that LMET graduates seem unclear or do not 
know what is expected [performance] of them, as well as their 
subordinates' similar concerns, may have implications beyond LMET. This 
hypotheses should be a first priority to test, using either additional 
research or in the evaluation plan. While it may only be conjecture, 
this and other communications concerns may well hold a key to increased 
performance and satisfaction within Navy commands. 
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e. Graduate Use of LMET Student Journal 

This issue is two-fold in nature depending upon the 
determination of the first concern, which is, what is the purpose of 
the student journal? LMET designers would contend that the student 
journal is a tool to be used by the graduate to help evaluate and 
reinforce actual leadership performance and self-made goals outside of 
the classroom. Additionally, it could serve to rearouse the LMET 
competencies. As such it should reinforce skills and behaviors that 
exemplify the LMET competencies while helping to correct those that do 
not. Dr. Winter further explains that the student journal and its 
personal student log, is the personal written record of an individual's 
strengths and weaknesses, set against the Navy's standard competencies 
that are associated with excellence and superior leadership 
performance.[WINTER,1979,pp.11-12] The first issue, then would be, to 
explore the purpose of the LMET student journal. If it is determined 
to be a beneficial tool to be used by the graduate, then, the second 
issue would be to determine when, why, and how is it used. The data do 
indicate that student use of the journal is minimal. 

f. Retention 

The issue of retention is of obvious importance to the 
Navy, but also pertains to LMET. This issue involves two underlying 
thoughts. The first is, does LMET training have any effect on the 
graduate's retention? The senority aspect of this may involve an 
examination of LMET's effect on junior personnel [division officers, 
and first class petty officers] as opposed to its effect on more senior 
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personnel [chief petty officers and department heads]. Should further 
investigation reveal that LMET training has a positive impact on re¬ 
tention, it may well indicate a need for increased emphasis on other 
levels of personnel. The second thought is investigating the effect of 
the graduates training on subordinates in relation to retention. 
Again, if further examination indicates a positive impact on retention 
by LMET graduates, it would assist in justifying the cost of the 
program. 

g. Reinforcement of LMET Competencies 

A final issue is the need for reinforcement of LMET 
competencies as perceived by the graduates. The pilot study data 
indicates that LMET competencies are more likely to be practiced when 
reinforced within the command [specifically by supervisors]. This 
issue of internal reinforcement was also evident from several LMET 
graduates who commented that active discouragement of LMET "new ideas" 
not only discouraged them, but frustrated their efforts in leadership 
and management. Internal reinforcement of the LMET competencies and 
the resultant effect on the graduate should be furthered explored. 

An additional related area requiring research is in the 
area of external support for LMET training. This support or lack of 
support outside of a command may provide some insight into the notion 
that LMET ends upon graduation from the school. This hypothesis could 
be tested during the full-scale evaluation as to whether external 
support or lack of support in fact affects the application of LMET 
competencies by the graduate. 
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These seven issues, while not all inclusive, should 

provide sufficient guidance to program managers and decision makers to 
design and conduct a meaningful evaluation of the LMET program. 

C. CONCLUDING REMARKS 

This thesis has emphasized the central point that the pilot study 
was not an evaluation of the LMET program, but rather the preliminary 
step to overall program evaluation to determine relevant issues and 
investigate a methodology to confront them. As a result, it has 

uncovered issues that require further investigation through research or 
within a comprehensive full-scale evaluation. 

Navy leaders and managers at all levels have a real need to know 
the effectiveness of the LMET program, that is, what it can or cannot 
do, as well as how to best support it. In times of austere funding and 

scarce resources, only those programs that can demonstrate their 

usefulness should justify the support of those in control. The clear 
way to demonstrate this usefulness is to conduct a program evaluation. 
This can identify weaknesses requiring correction as well as strengths 
that need re-emphasizing while justifying support for the program. 

The pilot study concept is a useful tool for program managers and 
decision makers to employ prior to committing expensive resources to an 
overall evaluation. The measurement of effectiveness is at best a 
tenuous business. The pilot study has explored this issue to help 
develop a viable method to address effectiveness. 

The flexibility afforded by a pilot study was evident in identify¬ 
ing relevant Issues which were unknown. Insights gained on the LMET 










program, as the pilot study progressed were used to update and modify 
its direction. In this way, new issues were explored for relevance 
almost immediately. A large scale evaluation would have found these 
constant changes to be very frustrating and costly. For the pilot 
study this flexibility was gained at no extra cost. The bottom line 
then becomes relevant issue identification at low cost. It must be 
remembered however, that issues identified by the pilot study should be 
considered unvalidated hypotheses to be further researched and explored 
or perhaps tested within an overall program evaluation. 

A final aspect of the pilot study which cannot be over emphasized, 
is its potential as a learning device for program managers. If 
involved in the pilot study, program managers can not help but gain a 
sharp insight into their programs. This insight can help focus their 
attention onto the relevant issues that need to be addressed in a full- 
scale evaluation. Furthermore, the knowledge gained by program 
managers can make it easier to monitor the progress of the larger 
evaluation. 

The pilot study provides a method whereby program managers can 
quickly and efficiently become aware of relevant issues. More import¬ 
antly, it can be used to provide direction and guidance for a meaning¬ 
ful program evaluation, prior to committing extensive resources. An 
effective evaluation strategy is the key to healthy program 
development. 
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APPENDIX A 


LMET GRADUATE SELF-REPORT OF ACTIVITIES AND RESULTS 23 


1. How long ago did you attend LMET? Did you attend prior to the 
command? 

2. Has there been any change in how you do your job, how you lead and 
manage others, after taking the LMET course? [Probe for specific 
examples] 

3. Has there been any change in your own career or career plans after 
taking the LMET course? [e.g. promotion, schools, PQS, retention 
intentions] [Probe for specifics] 

4. How is your performance on your job measured? [Probe for relevant 
indicies, rewards, task accomplishment, managerial ability, 
professional competence, etc.] Has there been any change in these 
job performance measures? What specifically? 

5. Has there been any change in your unit? 

- In job performance measures (e.g. inspections passed, producti¬ 
vity indicies, PMS accomplishment rate, accidents, etc.]. 

What specifically? 

- In satisfaction measures? [e.g. retention rates, NJP's, sick 
calls, UA's, etc.] 

6. What goals did you set in the LMET course? How have you completed 
these goals? Progress? Action steps completed? 

7. Have you ever used the student journal since you left the course? 
For what reason? [Probe for specifics] 

8. What was your level of participation in the LMET course [low, 
medium, high]? 

- In class 

- Homework 

9. Would you send anyone to LMET school? 


^^The LMET graduate was asked these 9 questions. Questions 2-8 were 
provided by McBer and Company. 
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LMET GRADUATE STIMULUS QUESTIONS 


24 


1. When you are faced with a new task, how do you go about accomplish, 
i ng i t? 

2. In your work center, how do you assign duties and responsibilities? 
What kind of paperwork do you do? [In relation to those assign, 
ments] 

3. What type of roadblocks do you run into on your job, and what do you 
do about them? 


4. What do you do if the work of your subordinates is worse than you 
expected? 

5. What do you do if the work of your subordinates is better than you 
expected? 


6. How do you go about getting someone to buy your idea about a better 
way of doing things? 

7. Tell me of a time when you identified a problem before anyone else 
did? How did you do this? 


8. How do you go about getting someone settled in who is new to the 
work center? [What is expected of him?] 

9. What happens to your work center when you are not there? If you 
took two weeks of emergency leave what would you expect to find upon 
your return? Why? 

10. How do you keep cool when someone is pulling your chain? 


2 ^The LMET graduate was asked these 10 questions. All questions 
were provided by McBer and Company with the exception of part 2 to 
question 9. 
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SUPERVISOR QUESTIONS VALIDATING LMET GRAOUATE'S ACTIVITIES 

AND RESULTS 25 


1. Are you an LMET graduate? How long ago did you graduate? 

2. How would you describe the way in which this subordinate [division 
officer, leading chief, work center supervisor] leads and manages 
others? Could you give me specific examples? 

3. Has there been any change in your subordinates career [e.g. 
promotions, schools, PQS, accomplishments] or career plans [e.g. 
re-enlistment, extensions, service obligations] while under the 

supervision, leadership and management of this _[division officer, 

leading chief, work center supervisor]? What specifically? What 
has he done? 

4. How is your subordinate's [division officer, leading chief, work 
center supervisor] performance measured? [Probe for specifics] 

Have you changed any of these measures? 

5. Has there been any changes in your unit as related to: 

- job performance measures [e.g. inspections, productivity, 
accidents, PMS accomplishments] 

- satisfaction measures [e.g. retention rates, NJP's, sick call, 
absentism] What, specifically? [Probe for examples] 

6. What do you like or admire most about his performance onboard? 

Least? [Probe for specifics] 

7. Would you work for him? Why or why not? 

8. What would you change in him if you could? Why? 

9. Would you send anyone to LMET school? Why? 

10. Is there anything else you might like to add in regards to this 
individual or LMET? 


25 The supervisor to the LMET graduate was asked those 10 
questions. Questions 2-5 were provided by McBer and Company. 
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SUBORDINATE QUESTIONS VALIDATING LMET GRADUATE'S ACTIVITIES 

AND RESULTS 26 


1. Are you a LMET graduate? How long ago did you graduate? 

2. How would you describe the way in which your boss [division 
officer, leading chief, work center supervisor] leads and 
manages others? Could you give me specific examples? 

3. Has there been any change in your own career [e.g. promotions, 
schools, PQS accomplishments] or career plans [e.g. re.enlist¬ 
ment, extensions, service obligations] while under the leader, 
ship and management of your boss [division officer, leading 
officer, work center supervisor]? What specifically? What has 
he done? 

4. How is your performance on the job measured by your boss 
[division officer, leading chief, work center supervisor]? 
[Probe for specific indicies] Have you noticed any changes in 
these performance measurements? 

5. Has there been any changes in your unit as related to: 

. job performance measures [e.g. inspections passed, 
productivity, accidents, PMS accomplishments] 

. satisfaction measures [e.g. retention rates, NJP's, sick 
calls, absentism] What, specifically? [Probe for examples] 

6. What do you like or admire most about his performance onboard? 
Least? Can you give specifics? 

7. Would you work for him again? Why or why not? 

8. What would you change in him if you could? Why? 

9. Would you like to attend LMET school if you could? What do you 
think you would get out of it? Why? 

10. Is there anything else you would like to tell me in regards to 
your boss or LMET? 


^ 6 The subordinate to the LMET graduate was asked these 10 
questions. Questions 2.5 were provided by McBer and Company. 
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